Systems and methods for automatic detection of spills

ABSTRACT

Systems and methods for automatic detection of spills are disclosed. In some exemplary implementations, a robot can have a spill detector comprising at least one optical imaging device configured to capture at least one image of a scene containing a spill while the robot moves between locations. The robot can process the at least one image by segmentation. Once the spill has been identified, the robot can then generate an alert indicative at least in part of a recognition of the spill.

COPYRIGHT

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent files or records, but otherwise reserves all copyrightrights whatsoever.

BACKGROUND Technological Field

The present application relates generally to robotics, and morespecifically to systems and methods for automatic detection of spills.

Background

Presently, spillage of water and other chemicals can pose injury risksfor nearby people. For example, people can slip on spills and injuretheir arms, legs, or other body parts. Spills can be especiallydangerous in store and warehouse environments where there can be highfoot traffic. Indeed, every year, slip-and-fall injuries send millionsof people to the hospital with an annual direct cost in the billions ofdollars. Moreover, these slip-and-fall injuries also result in deathsand millions of lost work days a year. Also, spillage of water and otherchemicals can also cause damage to surrounding surfaces and items whenthey spread, and can ruin clothing and other items. For example, storeitems can be damaged and/or carpet destroyed by spills.

In some cases, such spillage can occur when vessels containing thoseliquids are knocked over, such as by a customer or employee. Spills canalso occur during cleanings that involve liquids. For example, floorscrubbers use water and/or other chemicals to clean floors. In somecases, the water and/or other chemicals can leak or otherwise be left onfloors, creating hazards and potentially damaging surfaces and items.

Currently, methods of detecting spills often rely on human inspection,where persons who happen to come across those spills clean up (e.g.,mop) those spills or bring the spill to the attention of the properperson. Not only can these methods be inefficient, but many spills cango undetected for substantial amounts of time. Accordingly, there is aneed for improved systems and methods for detection of spills.

SUMMARY

The foregoing needs are satisfied by the present disclosure, whichprovides for, inter alia, apparatus and methods for spill detection.Example implementations described herein have innovative features, nosingle one of which is indispensable or solely responsible for theirdesirable attributes. Without limiting the scope of the claims, some ofthe advantageous features will now be summarized.

In some implementations a spill detector is disclosed. In some cases,the spill detector can be coupled to a robot. Where the spill detectoris attached to the robot, the spill detector can detect spills as therobot moves. Whether the spill detector is attached to the robot or not,when spills are detected, the spill detector can perform actions, suchas stopping the robot (and/or a system of the robot), alerting a user,and/or ignoring the spill.

In a first aspect, a robot is disclosed. In one exemplaryimplementation, the robot includes: an actuator configured to move therobot between locations; a spill detector comprising at least oneoptical imaging device configured to capture at least one image of ascene containing a spill while the robot moves between locations; and aprocessor configured to identify the spill in the at least one image andgenerate an alert indicative in part of a recognition of the spill.

In one variant, the optical imaging device is an infrared camera and theat least one image is a thermal image.

In another variant, the robot further includes a temperature adjusterconfigured to change the temperature of the scene containing the spill.In another variant, the temperature adjuster is at least one of anexhaust and a fan.

In another variant, the processor of the robot is further configured todetermine a confidence in the identification of the spill. In anothervariant, the confidence is determined based at least in part on Bayesianstatistical models.

In another variant, the robot further comprises a sensor configured todetect at least one of reflectance properties, emission properties,electrical properties, noises, and friction of the scene. In anothervariant, the confidence is based at least in part on information fromthe sensor and the at least one image.

In another variant, the processor is further configured to generate acolor image having colors based at least in part on thermal values of asegment of the at least one image, and determine if the colors areindicative at least in part of the spill.

In another variant, the robot further comprises a floor cleaning system.

In a second aspect, a method for detecting spills is disclosed. In oneexemplary implementation, the method includes: generating a first imageof a first scene at a first location that contains a spill; generating asecond image of a second scene at a second location that contains nospills; segmenting the first image to detect the spill from at leastthermal values in a segment of the first image; identifying the spill;and generating an alert indicative at least in part of theidentification of the spill.

In one variant, the method further includes adjusting the temperature ofthe first scene while generating the first image.

In another variant, the method further includes determining a confidencein the identified spill, wherein the generated alert is furtherindicative of the confidence.

In another variant, the method further includes sensing at least one ofreflectance properties, emission properties, electrical properties,noises, and friction of the first scene.

In another variant, the method further includes determining a confidencein the identified spill based at least in part on the segmentation ofthe first image and the sensed at least one of reflectance properties,emission properties, electrical properties, noises, and friction of thefirst scene, wherein the generated alert is further indicative of theconfidence.

In another variant, the method further includes receiving an actioncommand in response to the generated alert and performing an action inresponse to the action command.

In a third aspect, a non-transitory computer-readable storage medium isdisclosed. In one exemplary implementation, the non-transitorycomputer-readable storage medium has a plurality of instructions storedthereon, the instructions being executable by a processing apparatus tooperate a spill detector, the instructions configured to, when executedby the processing apparatus, cause the processing apparatus to: generatea first image of a first scene at a first location that contains aspill; generate a second image of a second scene at a second locationthat contains no spills; segment the first image to detect the spillfrom at least thermal values in a segment of the first image; identifythe spill; and generate an alert indicative at least in part of theidentification of the spill.

In one variant, the instructions further cause the processing apparatusto adjust the temperature of the first scene while generating the firstimage.

In another variant, the instructions further cause the processingapparatus to determine a confidence in the identified spill, wherein thegenerated alert is further indicative of the confidence.

In another variant, the instructions further cause the processingapparatus to sense at least one of reflectance properties, emissionproperties, electrical properties, noises, and friction of the firstscene.

In another variant, the instructions further cause the processingapparatus to determine a confidence in the identified spill based atleast in part on the segmentation of the first image and the sensed atleast one of reflectance properties, emission properties, electricalproperties, noises, and friction of the first scene, wherein thegenerated alert is further indicative of the confidence.

In another variant, the instructions further cause the processingapparatus to receive an action command in response to the generatedalert and perform an action in response to the action command.

In a fourth aspect, a spill detector is disclosed. In one exemplaryimplementation, the spill detector includes: one or more sensorsconfigured to generate data indicative of a spill when a spill ispresent; and a processor configured to determine a confidence that aspill has been detected from the generated data.

In one variant, the one or more sensors include a camera. In anothervariant, the one or more sensors include at least one of a microphone, alight meter, a dynamometer, a fluorescence detector, a fluorescenceimager, a capacitance meter, a voltmeter, a multimeter, an oscilloscope,an ohmmeter, and an ammeter.

In another variant, the processor is further configured to generate acommand based at least in part on the determined confidence. In onevariant, the command is a stop command. In one variant, the commandinstructs the spill detector to send a request to a user interface forfurther instructions.

In a fifth aspect, a robot that performs an action in response to aspill is disclosed. In one exemplary implementation, the robot includesan actuator configured to move the robot between locations; a spilldetector comprising at least one optical imaging device configured tocapture at least one image of a scene containing a spill while the robotmoves between locations; and a processor configured to generate anaction command based at least in part on the at least one image.

In one variant, the robot further includes a temperature adjusterconfigured to changes the temperature of the scene containing the spill.

In another variant, the action command is a stop command that stops therobot from moving between locations.

In a sixth aspect, a system for spill detection is disclosed. In oneexemplary implementation, the system includes a spill detectorcommunicatively coupled to a server. The server is communicativelycoupled to a robot and one or more access points.

In a seventh aspect, a control center is disclosed. In one exemplaryimplementation, the control center is communicatively coupled to a spilldetector through a server. The control center remotely sends commands toa robot based at least in part on spills detected by the spill detector.

These and other objects, features, and characteristics of the presentdisclosure, as well as the methods of operation and functions of therelated elements of structure and the combination of parts and economiesof manufacture, will become more apparent upon consideration of thefollowing description and the appended claims with reference to theaccompanying drawings, all of which form a part of this specification,wherein like reference numerals designate corresponding parts in thevarious figures. It is to be expressly understood, however, that thedrawings are for the purpose of illustration and description only andare not intended as a definition of the limits of the disclosure. Asused in the specification and in the claims, the singular form of “a”,“an”, and “the” include plural referents unless the context clearlydictates otherwise.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosed aspects will hereinafter be described in conjunction withthe appended drawings, provided to illustrate and not to limit thedisclosed aspects, wherein like designations denote like elements.

FIG. 1 is a side elevation view of an example robot having a spilldetector in accordance with some implementations of the presentdisclosure.

FIG. 2A illustrates various side elevation views of exemplary body formsfor floor scrubbers in accordance with some principles of the presentdisclosure.

FIG. 2B illustrates various side elevation views of exemplary body formsfor a robot in accordance with principles of the present disclosure.

FIG. 3 is a functional block diagram of an exemplary robot in accordancewith some implementations of the present disclosure.

FIG. 4 is a functional block diagram of an exemplary spill detectorcommunicatively coupled to a server in accordance with someimplementations of the present disclosure.

FIG. 5 is an exemplary user interface for alerting a user of a detectedspill in accordance with principles of the present disclosure.

FIG. 6 is a side elevation view of an exemplary spill detector imaging aspill on a surface in accordance to some principles of the presentdisclosure.

FIGS. 7A-7E are exemplary thermal images taken of a spill by a spilldetector that includes an infrared camera in accordance to someimplementations of the present disclosure.

FIG. 8 is a side elevation view of an exemplary fan configured to blowair onto a spill in accordance with some implementations of the presentdisclosure.

FIG. 9 is a side elevation view of an exemplary heating apparatus thatcan heat a spill in accordance with some implementations of the presentdisclosure.

FIGS. 10A-10C are exemplary thermal images taken when an example heatingunit heats a spill in accordance with some implementations of thepresent disclosure.

FIG. 11 is a process flow diagram of an exemplary method for detecting aspill in accordance with some implementations of the present disclosure.

FIG. 12 is a process flow diagram of an exemplary method for detecting aspill where a robot can ask for user assistance in identifying the spillin accordance with some implementations of the present disclosure.

FIG. 13 is a process flow diagram of an exemplary method for detecting aspill where a robot can adjust behaviors based on feedback in accordancewith some implementations of the present disclosure.

FIG. 14 is a process flow diagram of an exemplary method for detectingspills in accordance with principles of the present disclosure.

All Figures disclosed herein are © Copyright 2016 Brain Corporation. Allrights reserved.

DETAILED DESCRIPTION

I. Overview

Various aspects of the novel systems, apparatuses, and methods disclosedherein are described more fully hereinafter with reference to theaccompanying drawings. This disclosure can, however, be embodied in manydifferent forms and should not be construed as limited to any specificstructure or function presented throughout this disclosure. Rather,these aspects are provided so that this disclosure will be thorough andcomplete, and will fully convey the scope of the disclosure to thoseskilled in the art. Based on the teachings herein, one skilled in theart should appreciate that the scope of the disclosure is intended tocover any aspect of the novel systems, apparatuses, and methodsdisclosed herein, whether implemented independently of, or combinedwith, any other aspect of the disclosure. For example, an apparatus canbe implemented or a method can be practiced using any number of theaspects set forth herein. In addition, the scope of the disclosure isintended to cover such an apparatus or method that is practiced usingother structure, functionality, or structure and functionality inaddition to or other than the various aspects of the disclosure setforth herein. It should be understood that any aspect disclosed hereincan be implemented by one or more elements of a claim.

Although particular aspects are described herein, many variations andpermutations of these aspects fall within the scope of the disclosure.Although some benefits and advantages of the preferred aspects arementioned, the scope of the disclosure is not intended to be limited toparticular benefits, uses, and/or objectives. The detailed descriptionand drawings are merely illustrative of the disclosure rather thanlimiting, the scope of the disclosure being defined by the appendedclaims and equivalents thereof.

The present disclosure provides for improved systems and methods fordetection of spills. As used herein, a robot can include mechanical orvirtual entities configured to carry out complex series of actionsautomatically. In some cases, robots can be machines that are guided bycomputer programs or electronic circuitry. In some cases, robots caninclude electro-mechanical components that are configured fornavigation, where the robot can move from one location to another. Suchnavigating robots can include autonomous cars, floor cleaners, rovers,drones, carts, and the like.

As referred to herein, floor cleaners can include floor cleaners thatare manually controlled (e.g., driven or remote control) and/orautonomous (e.g., using little to no user control). For example, floorcleaners can include floor scrubbers that a janitor, custodian, or otherperson operates and/or robotic floor scrubbers that autonomouslynavigate and/or clean an environment.

Detailed descriptions of the various implementations and variants of thesystem and methods of the disclosure are now provided. While manyexamples discussed herein are in the context of robotic floor cleaners,it will be appreciated that the described systems and methods containedherein can be used in other robots. Myriad other example implementationsor uses for the technology described herein would be readily envisagedby those having ordinary skill in the art, given the contents of thepresent disclosure.

Advantageously, the systems and methods of this disclosure at least: (i)provide for automatic detection of spills; (ii) enable robotic detectionof spills; (iii) reduce or eliminate injuries and property damagethrough early spill detection; (iv) enable automatic robotic cleaning bydetecting and cleaning spills; (v) reduce or eliminate false positiveand/or false negative detection of spills; and (vi) enhance the abilityof spills to be detected by off-the-shelf components such as cameras.Other advantages are readily discernable by one having ordinary skillgiven the contents of the present disclosure.

As used herein, spills (and/or spillage) can include liquids and/orpartial liquids, such as water and/or other chemicals. Such otherchemicals include any type of chemical that may spill on a floor in aparticular environment. For example, in a grocery store, chemicals caninclude aqueous solutions, honey, milk, mustard, ketchup, beverages,bodily fluids, oil, butter, ice, candy, cleaners (e.g., cleaning fluid,floor wax, disinfectants, etc.), and/or other chemicals. In a warehouse,chemicals can include aqueous solutions, grease, oil, cleaners (e.g.,cleaning fluid, floor wax, disinfectants, etc.), industrial waste,coolant, etc.

The spills can be on a surface, such as a floor. Surfaces can includeany known surfaces, including those used as flooring in stores orwarehouses. For example, surfaces can comprise wood (e.g., engineered,synthetic, hardwood, etc.), bamboo, vinyl, concrete, ceramic tile,linoleum, porcelain tile, laminate, cork, stone, aluminum, metals,steel, epoxy, and other materials. In many cases, surfaces can includematerials in a solid state.

Accordingly, because of their different chemical make-ups and differentphysical states, the spills and surfaces can have differentproperties/characteristics as compared to each other. Some examples willbe briefly mentioned here, but later discussed in more detail insections II and III of this disclosure.

By way of illustration of some of the differences inproperties/characteristics, the spills can have different electricalproperties, such as conductance, impedance, and capacitance, from thesurface. The spills can also have different coefficients of frictionthan the surface.

The spills can also have different thermal properties as well. Forexample, many spills can be subject to physical phenomenon such asevaporative cooling (e.g., of volatile liquids), adiabatic expansion,Joule Thomson effects, and/or other thermodynamic effects of liquids,gases, or liquid-gas mixtures. As a result, many spills (e.g., water,aqueous solutions, oil, solvents, fuels, etc.) may be cooler than thesurfaces on which they reside due to the aforementioned physicalphenomenon. The temperature of the spill may also differ from that ofthe floor because the substance spilled may be originally at a differenttemperature, intentionally (e.g., to help detect the spill and/or toprevent spoilage) or unintentionally. For example, a person havingordinary skill in the art should appreciate that the thermal image of aspill of a warm or hot cleaning fluid would look quite different from aspill whose cleaning fluid is originally at room temperature or belowroom temperature.

Spills can also have different reflectance and/or emission of light. Forexample, some spills may be more reflective than their correspondingsurfaces. As a result, incident light (e.g., light from fixtures,sunlight, or any other light source) may reflect more from the spillsthan the surfaces. Spills of different temperature can also emitdifferent amounts of heat. Spills can also have differentultraviolet-induced florescence, where some spills can have uniqueflorescent properties when exposed to ultraviolet (“UV”) radiationand/or other radiation. Spill may have different reflectance and/oremission properties. For example, at or substantially near Brewster'sangle, reflected light can be at least partially polarized.

Despite having these different properties, detection of spills can stillhave challenges. For example, it may be desirable for a mobile robot,such as a robot that can navigate an environment, to locate and/or treatspills. However, some spills take time before their thermal propertiescause them to change temperature from the surface on which they aredeposited. Accordingly, where the robot relies at least in part onthermal differences, the robot may miss a spill if it passes it toosoon.

As another example, floor cleaners, such as floor scrubbers, use waterand/or other cleaners to clean a surface, such as a floor. In somecases, the water and/or other cleaners can be left on the floor asspills.

By way of illustration, FIG. 1 illustrates robot 100, which can be afloor scrubber having spill detector 112. Robot 100 has tanks 104, whichcan hold water and/or cleaning chemicals (e.g., detergent). As robot 100travels, an amount of water and cleaning chemicals is distributed to thefloor (e.g., a surface) through tube 106. Brush 108 then scrubs thefloor using the water and cleaning chemicals. Squeegee 118 wipes thedirty water and cleaning chemicals as a scrub vacuum fan, using tube 114to remove dirty water and cleaning chemicals from the floor. Steeringwheel 102 can be used to control robot 100, but in some implementations,robot 100 may be configured to navigate through remote control orautonomously. Robot 100 can have wheels, such as wheel 110. These wheelscan be coupled to an actuator 120, which is configured to cause thewheels to move and propel robot 100 forward. In this way actuator 120can move robot 100 from one location to another location.

Robot 100 can include spill detector 112 and temperature adjuster 116.Temperature adjuster 116 can be used to change the temperature of ascene imaged by spill detector 112. In some implementations, spilldetector 112 can be used to detect spills, such as dirty water andcleaning chemicals not wiped and/or vacuumed. Such dirty water andcleaning chemicals can, in some cases, be left in the floor when thereare malfunctions, mechanical failures, and/or any other problems withthe cleaning system of robot 100, including with tank 104, tube 106,actuator 120, tube 114, squeegee 118, or any other component orsubcomponent of robot 100.

By way of illustration, tank 104 could have a leak, wherein excessiveliquid (e.g., water) flows to the floor such that not all the water canbe and/or is wiped and/or vacuumed. This can leave spills on the floor.As another example, a regulator of tube 106 and/or tank 104 couldmalfunction and excessive water could be put on the floor, leading tospills forming. As another example, actuator 120 can cause robot 100 tostop (intentionally or through malfunction). If water from tank 104 andtube 106 continues to be distributed to the floor, a spill can form andpossibly spread.

As another example, tube 114 can disconnect from robot 100 or squeegee118 (e.g., in cases where they are connected). This can cause water onthe floor to not be vacuumed, leading to spills being left behind robot100.

As another example, squeegee 118 can dislodge or otherwise be displacedin a way that it may not effectively wipe water from robot 100. In thesecases, squeegee 118 may not be wiping the floor effectively leading tospills being left behind robot 100.

Any of the aforementioned examples, alone or in combination, can causerobot 100 to leave a spill. A person having ordinary skill in the artshould appreciate that there can be any number of other reasons whyrobot 100 could leave a spill, and this disclosure is not limited to anyparticular ones. Moreover, there can be any other of other reasons forspills, such as items falling off shelves in grocery stores, peoplespilling beverages or cleaning solutions, etc. As previously mentioned,these spills can create hazards for people and objects. If not cleanedup and/or otherwise addressed, the chances of injury and/or propertydamage increases.

Robot 100 can have a plurality of sides, including front side 122, backside 124, right side 126, and left side (not illustrated). A personhaving ordinary skill in the art should appreciate that robot 100 canhave other sides as well, corresponding to the surfaces of robot 100,which can vary by shape (e.g., rectangular, pyramidal, humanoid, or anydesigned shape). By way of illustration, front side 122 can bepositioned on the forward-facing side of robot 100, where theforward-facing side is forward in the direction of forward movement ofrobot 100. Back side 124 can be positioned on the backward-facing sideof robot 100, where the backward-facing side is the side facing insubstantially the opposite direction of the forward facing side. Rightside 126 can be the right-hand side relative to front side 122, and leftside (not illustrated) can be the left-hand side relative to front side122.

A person having ordinary skill in the art should appreciate that robot100 can have a number of different appearances/forms, even if robot 100is a floor scrubber. FIG. 2A illustrates six example body forms for afloor scrubber. These are non-limiting examples meant to furtherillustrate the variety of body forms, but not to restrict robot 100 toany particular body form or even to a floor scrubber. Example body form202 has an upright shape with a small frame where a user can push thebody form 202 in order to clean a floor. In some cases, body form 202can have motorized propulsion that can assist a user in cleaning, butcan also allow for autonomous movement of body form 202. Body form 204has a larger structural shape than body form 202. Body form 204 can bemotorized enabling it to move with little to no user exertion upon bodyform 204 besides steering. The user may steer body form 204 as it moves.Body form 206 can include a seat, pedals, and a steering wheel, where auser can drive body form 206 like a vehicle as body form 206 cleans.Body form 208 can have a shape that is larger than body form 206 and canhave a plurality of brushes. Body form 210 can have a partial or fullyencased area where a user sits as he/she drives body form 210. Body form212 can have a platform where a user stands while he/she drives bodyform 212.

Further still, as described in this disclosure, robot 100 may not be afloor scrubber at all. For additional illustration, and withoutlimitation, FIG. 2B illustrates some additional examples of body formsof robot 100. For example, body form 214 illustrates an example whererobot 100 is a stand-up shop vacuum. Body form 216 illustrates anexample where robot 100 is a humanoid robot having an appearancesubstantially similar to a human body. Body form 218 illustrates anexample where robot 100 is a drone having propellers. Body form 220illustrates an example where robot 100 has a vehicle shape having wheelsand a passenger cabin. Body form 222 illustrates an example where robot100 is a rover. Body form 224 illustrates an example where robot 100 isa shopping cart. Body form 224 can be motorized to operate autonomously.

FIG. 3 illustrates a functional block diagram of example robot 100 insome implementations. Robot 100 can be a robotic floor cleaner. Robot100 can include controller 304, memory 302, user interface 318,actuators 320, temperature adjuster 116, spill detector 112, as well asother components and subcomponents not illustrated.

Controller 304 can control the various operations performed by robot100. Controller 304 can include one or more processors (e.g.,microprocessors) and other peripherals. As used herein, processor,microprocessor, and/or digital processor can include any type of digitalprocessing device such as, without limitation, digital signal processors(“DSPs”), reduced instruction set computers (“RISC”), general-purpose(“CISC”) processors, microprocessors, gate arrays (e.g., fieldprogrammable gate arrays (“FPGAs”)), programmable logic device (“PLDs”),reconfigurable computer fabrics (“RCFs”), array processors, securemicroprocessors, specialized processors (e.g., neuromorphic processors),and application-specific integrated circuits (“ASICs”). Such digitalprocessors may be contained on a single unitary integrated circuit die,or distributed across multiple components (e.g., circuit dies).

Controller 304 can be operatively and/or communicatively coupled tomemory 302. Memory 302 can include any type of integrated circuit orother storage device configured to store digital data including, withoutlimitation, read-only memory (“ROM”), random access memory (“RAM”),non-volatile random access memory (“NVRAM”), programmable read-onlymemory (“PROM”), electrically erasable programmable read-only memory(“EEPROM”), dynamic random-access memory (“DRAM”), Mobile DRAM,synchronous DRAM (“SDRAM”), double data rate SDRAM (“DDR/2 SDRAM”),extended data output RAM (“EDO”), fast page mode RAM (“FPM”), reducedlatency DRAM (“RLDRAM”), static RAM (“SRAM”), “flash” memory (e.g.,NAND/NOR), memristor memory, pseudostatic RAM (“PSRAM”), etc. Memory 302can provide instructions and data to controller 304. For example, memory302 can be a non-transitory, computer-readable storage medium having aplurality of instructions stored thereon, the instructions beingexecutable by a processing apparatus (e.g., controller 304) to operaterobot 100. In some cases, the instructions can be configured to, whenexecuted by the processing apparatus, cause the processing apparatus toperform the various methods, features, and/or functionality described inthis disclosure. Accordingly, controller 304 can perform logical andarithmetic operations based on program instructions stored within memory302.

In some implementations, memory 302 can store a library 324 of imagesof, for example, spills. In some implementations, this library 324 caninclude images of spills with different compositions (e.g., water and/orother chemicals) in different lighting conditions, angles, sizes,distances, clarity (e.g., blurred, obstructed/occluded, partially offframe, etc.), colors, surroundings, etc. The images in library 324 canbe taken by a spill detector (e.g., spill detector 112 or any otherspill detector) or generated automatically, such as with a computerprogram that is configured to generate/simulate (e.g., in a virtualworld) library images of spills (e.g., which can generate/simulate theselibrary images entirely digitally or beginning from an actual image of aspill or substantially similar objects) from different lightingconditions, angles, sizes, distances, clarity (e.g., blurred,obstructed/occluded, partially off frame, etc.), colors, surroundings,etc. In some implementations, library 324 can include thermal images.Library 324 can be used to train controller 304 to identify spills inmany conditions will be discussed more at least with reference to FIG.11, as well as throughout this disclosure. The number of images inlibrary 324 can depend at least in part on one or more of the number ofavailable images of spills, the variability of the surroundingenvironment in which robot 100 will operate, the complexity of spills,the variability in appearance of spills, the type of chemicals that maybe in spills, and/or the amount of available storage space (e.g., inlibrary 324, memory 302, and/or on a server). For example, library 324can contain 1, 5, 10, 100, 1000, 10,000, 100,000, 1,000,000, 10,000,000,or any number of images of spills. In some implementations, library 324may be stored in a network (e.g., cloud, server, etc.) and may not besaved within memory 302. As yet another example, various robots (e.g.,that are associated with a manufacturer) can be networked so that imagescaptured by individual robots are collectively shared with other robots.In such a fashion, these robots are able to “learn” and/or share imagingdata in order to facilitate the ability to readily detect spills.

In some implementations, user interface 318 can be configured to enablea user to interact with robot 100. For example, user interfaces 318 caninclude touch panels, buttons, keypads/keyboards, ports (e.g., universalserial bus (“USB”), digital visual interface (“DVI”), Display Port,E-Sata, Firewire, PS/2, Serial, VGA, SCSI, audioport, high-definitionmultimedia interface (“HDMI”), personal computer memory cardinternational association (“PCMCIA”) ports, memory card ports (e.g.,secure digital (“SD”) and miniSD), and/or ports for computer-readablemedium), mice, rollerballs, consoles, vibrators, audio transducers,and/or any interface for a user to input and/or receive data and/orcommands, whether coupled wirelessly or through wires. User interface318 can include a display, such as, without limitation, liquid crystaldisplay (“LCDs”), light-emitting diode (“LED”) displays, LED LCDdisplays, in-plane-switching (“IPS”) displays, cathode ray tubes, plasmadisplays, high definition (“HD”) panels, 4K displays, retina displays,organic LED displays, touchscreens, surfaces, canvases, and/or anydisplays, televisions, monitors, panels, and/or devices known in the artfor visual presentation. In some implementations user interface 318 canbe positioned on the body of robot 100, such as including a screenand/or console located on robot 100. In some implementations, userinterface 318 can be positioned away from the body of robot 100, but canbe communicatively coupled to robot 100 (e.g., via communication unitsincluding transmitters, receivers, and/or transceivers) directly orindirectly (e.g., through a network, server, and/or a cloud). In somecases, user interface 318 can communicate to robot 100 through a server,such as server 400 as will be described with reference to FIG. 4 as wellas elsewhere throughout this disclosure. In some implementations, userinterface 318 can be located on one or more of access points 402A-N aswill also be described with reference to FIG. 4 as well as elsewherethroughout this disclosure.

The wireless connections and/or wireless coupling can include wirelesstransmissions configured to send/receive a transmission protocol, suchas BLUETOOTH®, ZIGBEE®, Wi-Fi, induction wireless data transmission,radio frequencies, radio transmission, radio-frequency identification(“RFID”), near-field communication (“NFC”), infrared, networkinterfaces, 3G (3GPP/3GPP2), high-speed downlink packet access(“HSDPA”), high-speed uplink packet access (“HSUPA”), time divisionmultiple access (“TDMA”), code division multiple access (“CDMA”) (e.g.,IS-95A, wideband code division multiple access (“WCDMA”), etc.),frequency hopping spread spectrum (“FHSS”), direct sequence spreadspectrum (“DSSS”), Personal Area Network (“PAN”) (e.g., PAN/802.15),worldwide interoperability for microwave access (“WiMAX”), 802.20,narrowband/frequency-division multiple access (“FDMA”), orthogonalfrequency-division multiplexing (“OFDM”), cellular (e.g., 3G, long termevolution (“LTE”) (e.g., LTE/LTE-A), time division LTE (“TD-LTE”),global system for mobile communication (“GSM”), etc.), analog cellular,cellular digital packet data (“CDPD”), satellite systems, millimeterwave or microwave systems, acoustic, and infrared (e.g., infrared dataassociation (“IrDA”)), and/or any other form of wireless datatransmission.

As used herein, networks, servers, and/or clouds can include networkinterfaces. Network interfaces can include any signal, data, or softwareinterface with a component, network, or process including, withoutlimitation, those of the FireWire (e.g., FW400, FW800, FWS800T, FWS1600,FWS3200, etc.), universal serial bus (“USB”) (e.g., USB 1.X, USB 2.0,USB 3.0, USB Type-C, etc.), Ethernet (e.g., 10/100, 10/100/1000 (GigabitEthernet), 10-Gig-E, etc.), multimedia over coax alliance technology(“MoCA”), Coaxsys (e.g., TVNET™), radio frequency tuner (e.g., in-bandor OOB, cable modem, etc.), Wi-Fi (802.11), WiMAX (e.g., WiMAX(802.16)), PAN (e.g., PAN/802.15), cellular (e.g., 3G,LTE/LTE-A/TD-LTE/TD-LTE, GSM, etc.), IrDA families, etc. As used herein,Wi-Fi can include one or more of IEEE-Std. 802.11, variants of IEEE-Std.802.11, standards related to IEEE-Std. 802.11 (e.g., 802.11a/b/g/n/ac/ad/af/ah/ai/aj/aq/ax/ay), and/or other wireless standards.

Wired coupling can include wired connections, such as any cable that hasa signal line and ground. For example, such cables can include Ethernetcables, coaxial cables, Universal Serial Bus (“USB”), FireWire, and/orany connection known in the art. Such protocols can be used by robot 100to communicate to internal systems (e.g., communications between anycomponents and/or subcomponents of robot 100) and/or external systems(e.g., computers, smart phones, tablets, data capture systems, mobiletelecommunications networks, clouds, servers, and/or the like).

Actuators 320 can include any system used for actuating. For example,actuators 320 can include driven magnet systems, motors/engines (e.g.,electric motors, combustion engines, steam engines, and/or any type ofmotor/engine known in the art), solenoid/ratchet system, piezoelectricsystem (e.g., an inchworm motor), magnetostrictive elements,gesticulation, and/or any actuator known in the art. Actuators 320 caninclude actuator 120, as described with reference to FIG. 1. In someimplementations, actuators 320 can include systems that allow movementof robot 100, such as motorize propulsion. For example, motorizedpropulsion can move robot 100 in a forward or backward direction, and/oraid in turning robot 100 left or right. By way of illustration, in thisway, in this way, actuators 320 can control if robot 100 is moving or isstopped and/or allow robot 100 to navigate from one location to anotherlocation.

Actuators 320 can also be configured to actuate other instruments ofrobot 100, such as turning on/off water, spraying water, turning on/offvacuums, moving vacuum hose positions, turning spill detector 112,turning on/off temperature adjuster 116, turning temperature adjuster116, and/or any other action. For example, actuators 320 can turn off/onthe distribution of water to the floor through tank 104 and tube 106 bycontrolling a valve (e.g., a mechanical and/or electrical valve) thatcan turn off/on water flow and/or a water system.

Spill detector 112 can include systems that can be used to detect spill306. In some implementations, spill detector 112 can includemachine-imaging, such as the machine imaging described in U.S. Pat. No.6,812,846 to Gutta et al., which is incorporated herein by reference inits entirety. Spill detector 112 can include sensors such as a photocamera, video camera, infrared camera, and other cameras. Spill detector112 can also include other sensors such as microphones, light meters,dynamometer, fluorescence detector, fluorescence imager, capacitancemeter, voltmeter, multimeter (e.g., a Digital Multimeter (“DMM”)),oscilloscope, ohmmeter, ammeter, etc. In some implementations, spilldetector comprises at least one optical imaging device configured tocapture at least one image of a scene containing a spill. As will bedescribed with reference to at least FIG. 11 and elsewhere throughoutthis disclosure, spill detector 112 can take images including imageswith spill 306 (or other spills) in view. From those images, spilldetector 112 can detect the presence of spills. A person having ordinaryskill in the art should appreciate that spill 306 can be any shape andis not limited to any particular illustration shown in this disclosure.Indeed, even the same spill 306 can take on different shapes over timeas spill 306 spreads, moves, etc. Spill 306 can be any spill describedin this disclosure, such as liquids and/or partial liquids, such aswater and/or other chemicals.

Spill detector 112 may not be physically located on robot 100. Forexample, in some cases, spill detector may be attached to a wall, shelf,ceiling, fixture, other shopping carts, furniture, etc. Spill detectorcan then be communicatively coupled to robot 100, such as using wirelessand/or wired coupling. Spill detector 112 can have its own controller(e.g., with a processor) and/or be operatively and/or communicativelycoupled to controller 304. Accordingly, processing described in thisdisclosure, including systems and methods relating to spill detection,can be performed in spill detector 112 and/or a controller such ascontroller 304.

In some implementations, spill detector 112 can be communicativelycoupled to a server, such as through wired and/or wireless connections.FIG. 4 illustrates a diagram where example spill detector 112 iscommunicatively coupled to example server 400. Spill detector 112 cansend statuses, commands, system errors, data, alerts, warnings,measurement data, summary data regarding measurements, informationindicative at least in part of spills, and/or other information relevantto the operation of spill detector 112 and the identification of (e.g.,indicating and/or showing the location of) spills. In someimplementations, server 400 can comprise a collection of hardware,software, services, and/or resources that can be invoked to instantiatea virtual machine, process, or other resource for a limited or definedduration, or an unlimited or undefined duration. Server 400 can also becalled a network, cloud, etc. Server 400 can be communicatively oroperatively coupled to a plurality of devices, systems, computers,and/or servers, including devices and/or servers that have access to theinternet. Server 400 may also process any data received from spilldetector 112. For example, server 400 can use at least in part datareceived from spill detector 112 and generate an alert (e.g., a message,notification, and/or any form of communication) indicating at least inpart whether a spill (e.g., spill 306) has been detected, the status ofspill detector 112, the location of a spill, current or past data fromspill detector 112, a command or indication of action that should betaken (e.g., action by a user, robot, and/or of access points402A-402N), and/or other information relevant to a reaction to a spill.

Robot 100 can also be communicatively coupled to a server 400, such asthrough wired and/or wireless connections. Where spill detector 112 isnot in direct communication with robot 100, robot 100 and spill detector112 can exchange data and/or other communications through server 400.Robot 100 can receive any of the aforementioned data from spill detector112 and/or processed data from spill detector 112 from server 400. Also,robot 100 can send to server 400 data and/or other communicationsincluding statuses, commands, system errors, data, alerts, warnings,measurement data, summary data regarding measurements, informationindicative at least in part of spills, and/or other information relevantto the operation of spill detector 112 and/or robot 100. This dataand/or other communications can be received from server 400 by spilldetector 112 and/or any of access points 402A-402N. Any of theaforementioned data and/or communications between robot 100 and server400 and/or spill detector 112 and server 400 can also be communicateddirectly between one or more of spill detector 112, robot 100, andaccess points 402A-402N.

Access points 402A-402N, can include devices, systems, and/or servers,such as, but not limited to, computers, mainframes, remote operatingcenters, mobile devices, tablets, smart phones, cells phones, personaldigital assistants, phablets, smart watches, set-top boxes, and/or anydevice with access to the internet and/or any network protocol. As usedherein the “N” in access points 402A-402N indicates at least in partthat there can be any number of access points, and this disclosure isnot limited to any particular number of access points, nor does thisdisclosure require any number of access points. Access points 402A-402Ncan be communicatively coupled to server 400, such as through wiredand/or wireless connections. Each of access points 402A-402N can sendand/or receive information to/from server 400. For example, each ofaccess points 402A-402N can send data and/or communications such asstatuses, commands, system errors, measurement data, alerts, warnings,and/or other data and/or communications. Through server 400, accesspoints 402A-402N can receive, for example, at least a portion of thedata and/or communications sent by spill detector 112 to server 400,processed data by server 400 (e.g., from the data received by server 400from spill detector 112), at least a portion of the data and/orcommunications sent by robot 100 to server 400, at least a portion ofthe data and/or communications sent by one or more of access points402A-402N, and/or any other data on server 400.

By way of illustrative example, access point 402A can include a computeror set of computers. In some cases the computers of access point 402 canbe part of a remote operations controller (“ROC”) and/or controlstation. In this role, access point 402 can be used to monitor and/orcontrol one or more of spill detector 112, robot 100, and/or any ofaccess points 402A-402N. Advantageously, access point 402A can be usedto monitor spill detector 112 and/or determine if there are any issues(e.g., if there are any spills). If there are any issues, access point402A can send alerts, commands, and/or other communications to robot100. For example, access point 402A can send a command and/or alert torobot 100 which can cause at least in part robot 100 to stop and/or turnoff its cleaning system (e.g., by turning off one or more actuators ofactuators 320), as will be described later in this disclosure withreference to FIG. 11 as well as elsewhere throughout this disclosure. Asanother non-limiting example, access point 402A can give robot 100navigation instructions, such as directing robot 100 to turn, go to aparticular location, and/or generally remote control robot 100.Similarly, access point 402B can include a mobile device that can beconfigured with similar monitoring and/or controlling capabilities asaccess point 402A.

FIG. 5 illustrates an example interface 500 that can be used to alert auser of a detected spill 306. Interface 500 can be any user interfacediscussed in this disclosure, such as user interfaces discussed withreference to user interface 318. As illustrated, the appearance ofinterface 500 is merely for illustrative purposes, and any number ofother appearances is contemplated. A person having ordinary skill in theart would appreciate that interface 500 can be adapted for display onany interface, such as any display discussed in this disclosure,including those displays discussed with reference to user interface 318.Interface 500 can show display 502. When a spill is detected, panel 504can display an alert indicative at least in part that a spill has beendetected. Other information can be displayed in addition or in thealternative to panel 504, including information relevant to robot 100and/or spill detector 112 such as statuses, commands, system errors,data, alerts, warnings, measurement data, summary data regardingmeasurements, information indicative at least in part of spills (e.g.,spill 306), and/or other information relevant to the operation of spilldetector 112 and identifying (e.g., indicating and/or showing thelocation of) spills.

Panel 506 can include any data measured by spill detector 112. Forexample, where spill detector 112 includes a camera, such as ared-green-blue (“RGB”) camera, photo camera, video camera, infraredcamera, and other cameras, panel 506 can show the camera image. By wayof illustration, the camera image can be a RGB camera image of spill306. As another example, the image can be an infrared image (e.g., athermal image), such as the infrared images that will be described withreference to FIGS. 7A-7E, 10A-10C, as well as elsewhere throughout thisdisclosure. A person having ordinary skill in the art should appreciateother images can also be displayed, such as images that comprise of dataassociated with pixels (e.g., where pixels correspond to locations in animaged scene). For example, the image can be a matrix that stores aplurality of values, such as one or more measurement values (e.g.,measurements, measured temperatures, relative temperatures, etc.),representative colors, luminance, chrominance, and/or otherdata/information. The images may not appear as a spill 306 would tohuman eyes. The images and/or panel 506 can also display other data ofspill detector 112, such as data from microphones, light meters,dynamometer, fluorescence detector, fluorescence imager, capacitancemeter, voltmeter, multimeter (e.g., a Digital Multimeter (“DMM”)),oscilloscope, ohmmeter, ammeter, etc.

Display 502 can also present a user with selectable options, such asoptions 508, 510, 512. Options 508, 510, 512 can allow a user to performan action in response to what is displayed in one or more of panels 504,506, such as an indication that a spill has been detected. For example,option 508 can be a stop option that tells robot 100 to stop. In thecase where robot 100 is a floor cleaner, such as a floor scrubber, thestop option can send a signal to robot 100 to actuate one or moreactuators 320, causing robot 100 to, for example, stop moving, turn offa water system and/or water flow, stop cleaning (e.g., stop a brushand/or cleaning system), etc. As another example, option 510 can be analert help option that sends a signal to robot 100 and/or a differentelectronic apparatus or device (e.g., one or more of access points402A-402N) to generate an alert about spill 306. For example, option 510can cause user interface 500 and/or robot 100 to send a short messageservice (“SMS”), text, email, or other communication to a viewer who cango clean up spill 306. Option 510 could also trigger an alert, such asan alarm, flashing light, sound, and/or any other way of gettingsomeone's attention to clean up spill 306. As another example, option512 can be an ignore option where the user tells robot 100 and/or spilldetector 112 to continue operation and ignore spill 306. In some cases,the ignore option can be indicative at least in part that spill 306 isnot an actual spill and/or is a false positive. In some cases, theignore option may be used when spill 306 is an actual spill, but is justnot a concern due to its size, location, timing (e.g., at night when noone would slip on it), was intentionally placed, and/or any othercharacteristic. In some cases, panel 506 can inform a user of what spill306 looks like so that the user can select one or more of options 508,510, 512, and/or perform any other action described in this disclosure.

FIG. 6 illustrates spill detector 112 imaging an example spill 306 onexample surface 604. As illustrated, spill detector 112 can have fieldof view 602. As previously described with reference to at least FIGS. 1,3, 4, as well as elsewhere throughout this disclosure, spill detector112 can be a component of and/or be connected to robot 100 in someimplementations. In other implementations, spill detector 112 can beseparate and/or not attached to robot 100, but can be communicativelycoupled to robot 100 and/or server 400.

In the case where spill detector 112 includes an infrared camera, spilldetector 112 can measure temperatures (e.g., in Celsius, Kelvin,Fahrenheit, etc.), or relative temperatures, within field of view 602.The infrared camera can be single beam or multi-beam. In some cases, theinfrared camera can be coupled to an actuator that enables it to view aplurality of scenes in field of view 602. As previously mentioned, anyspill detector 112 can similarly be coupled to an actuator.

By way of illustration, the infrared camera can detect infrared energy(e.g., heat) and convert that detected infrared energy into anelectronic signal, which can then be processed to produce what issometimes called a thermal image or heat map. The thermal image cancontain pixels, wherein each pixel represents a location of an imagedspace and has a value corresponding to a temperature (e.g., atemperature measurement or a relative temperature). For example, thetemperature can be a measured value (e.g., based at least in part ondetected infrared energy), wherein the infrared camera takesmeasurements at least in part in field of view 602 and then uses acalibration function (e.g., instantiated in software and/or hard-coded)that converts the measurements into a temperature reading. As anotherexample, the infrared camera can take measurements (e.g., based at leastin part on detected infrared energy) and represent the measurementsbased at least in part on their relative magnitude. For example, themeasurements can be luminance values used to color a thermal image,where different colors on a scale (e.g., having one or more colors inthe visible spectrum ordered in wavelength) indicate at least in partrelative colors, where colors closer to one color in the scalerepresents cooler temperatures and colors closer to another color in thescale represent warmer temperatures. In some implementations, thethermal image can be visualized (e.g., on a user interface and/or storedin memory) as a picture having colors that correspond to themeasurements at each pixel. In this way, a thermal image can be viewed.In some implementations, the thermal image may comprise a matrix (e.g.,an m×n matrix having m rows and n columns) where each cell of the matrixcan represent a pixel of the thermal image, and each cell of that matrixstores the corresponding measurement value (e.g., measurements, measuredtemperatures, relative temperatures, etc.). In some cases, a thermalimage can be a matrix that stores a plurality of values, such as one ormore measurement values (e.g., measurements, measured temperatures,relative temperatures, etc.), representative colors, luminance,chrominance, and other data/information.

In some implementations, where spill detector 112 includes a pluralityof cameras and/or other sensors, images can be 3D images. By way ofillustration, a first camera can take a first image at a first angle.This first image can be 2D having X₁-Y₁ dimensions (which can be mappedwith X₁, Y₁ coordinates). At substantially the same time, a secondcamera can take a second image at a second angle. This second image canbe 2D having X₂-Y₂ dimensions (which can be mapped with X₂, Y₂coordinates). Controller 304 can receive the first image and secondimage. In some cases, controller 304 can create a 3D image based atleast in part on X₁-Y₁ dimensions from the first image and a Z₁dimension calculated at least in part on the X₂-Y₂ dimensions of thesecond camera. In some implementations, the first camera can besubstantially orthogonal to the second camera and/or lie insubstantially the same horizontal plane. In those cases, the 3D map canbe generated in some implementations by taking the X₁-Y₁ dimensions ofthe first image and the X₂ dimension of the second image. However, insome cases, the first camera, the second camera, and/or any other sensormay not be substantially orthogonal and/or not and/or lie insubstantially the same horizontal plane to each other. In these cases,controller 304 can construct the three-dimensional map usingthree-dimensional reconstruction from line projections based at least inpart on the images taken from the first camera, the second camera,and/or any other sensor. In some cases, the first camera and secondcamera can be at least one of an RGB camera, IR camera, photo camera,video camera, etc. Where an IR camera is used, the first and secondimages can be thermal images.

II. Thermal Imaging to Detect Spills

FIGS. 7A-7E illustrates example thermal images 700A-700E taken ofexample spill 306 by example spill detector 112 that includes aninfrared camera. In these example thermal images 700A-700E, a visiblelight image outline is overlaid on the thermal image as an intensitymodulation pattern. It should be noted that this is for illustrativepurposes, and a person having ordinary skill in the art shouldappreciate that the images may or may not include the visible lightimage outline, where the visible-light modulation pattern may be absentwithout the visible light image outline.

Example spill 306 is represented as spill image 716A-716E in thermalimages 700A-700E, respectively. Similarly, surface 604 is represented assurface image 718A-718E in thermal images 700A-700E, respectively. Byway of illustration, thermal images 700A-700E were taken using aninfrared camera in an experiment using tap water that is substantiallysimilar in temperature to the ambient environment including surface 604,however, it should be understood by a person having ordinary skill inthe art that spill 306 can have different shapes and compositions, asdescribed in “I. Overview” as well as elsewhere throughout thisdisclosure. Also, spill 306 can be different temperatures, such aswarmer or cooler than the ambient environment including surface 604.Thermal images 700A-700E can include images that have values indicativeat least in part of reflectance (e.g., IR reflectance values) and/oremission at each pixel location, wherein the pixel locations correspondto a position in field of view 602.

Thermal image 700A of FIG. 7A includes spill image 716A of spill 306when spill 306 is a new spill, such as taken by spill detector 112substantially right after (e.g., within approximately 10 seconds) spill306 contacted surface 604. Thermal image 700B of FIG. 7B includes spillimage 716B of spill 306 approximately one minute after spill 306contacted surface 604. Thermal image 700C of FIG. 7C includes spillimage 716C approximately two minutes after spill 306 contacted surface604. Thermal image 700D includes spill 306 approximately after threeminutes after the spill occurred. Thermal image 700E includes spill 306approximately after five minutes after the spill occurred.

In some implementations, thermal images 700A-700E can be displayed on auser interface 318 where a user can select, e.g., by placing reticules704A-704E, a location at which the user desires to view a temperature asdisplayed on temperature display panels 702A-702E. As displayed on userinterface 318, thermal images 700A-700E can include bars 710A-710E,respectively. Bars 710A-710E can indicate the pixel brightness valuesdisplayed at locations within thermal image 700A-700E. In some cases,bars 710A-710E can include a range of values from a low value to a highvalue, wherein the low value corresponds to one color and the high valuecorresponds to another color, and a gradient of colors in-between. Byway of illustrative example, bar 710A has a low value of 16.7 degreesCelsius and a high value of 25.0 degrees Celsius. The low value of 16.7degrees Celsius is associated with a black color, whereas the high valueof 25.0 degrees Celsius is associated with a white color. Thetemperatures between 16.7 degrees Celsius and 25.0 degrees Celsius arerepresented by a continuous spectrum (e.g., a gradient) of colorsbetween the black and white, wherein the colors of pixels in thermalimage 700A that are closer to white are closer to the high value of 25.0degrees Celsius and the colors closer to black are closer to the lowvalue of 16.7 degrees Celsius. The other images 700B-700E havesubstantially similar bars 710B-710E and similarrepresentations/associations. Accordingly, a viewer of one or more ofthermal images 700A-700E on user interface 318 may be able to discernthe relative and/or approximate temperatures of any given pixel/locationon thermal images 700A-700E by comparing the colors as they appear atthose pixels/locations with the colors illustrated in bars 710A-700E.Other representations can also be made in thermal images, including ofthermal luminosity and chrominance, as well as spectral properties ofemitted, reflected, scattered, and/or absorbed radiation.

A person having ordinary skill in the art should appreciate that thermalimages 700A-700E can also be displayed on user interface 318 withouttemperature display panels 702A-702E and bars 710A-710E. Similarly,thermal images 700A-700E can be stored in memory 302 with or withouttemperature display panels 702A-702E and bars 710A-710E. In some cases,where thermal images 700A-700E are stored in memory 302 and not viewed,each pixel may or may not have an associated color. Rather, in someimplementations, temperatures, reflectance values, emission values,and/or other measurements can be associated with each pixel. Bad pixels(e.g., inaccurate or erroneous pixels) can be removed, and/or additionalimage processing can be applied.

As illustrated in thermal images 700A-700E, as time progressed after aspill event, spill 306 became more discernable as spill images 716A-716Ein the respective thermal images 700A-700E. For example, in thermalimage 700A, which can be taken right after spill 306 occurred, spillimage 716A may be difficult to discern, indicative at least in part thatspill 306 was a substantially similar temperature as surface 604. Edge706A appears as a different color indicating at least in part that edge706A was measured as a slightly cooler temperature than the portion ofsurface 604 imaged as surface image 718A. Center spill area 708A appearssubstantially similar in color to surface 604 because it issubstantially similar in temperature. It is possible that suchtemperature differences can be observable due at least in part toevaporative cooling, wherein the outer edges of spill 306 cool morerapidly than more center portions, such as center spill area 708A. Thisproperty may improve spill detection, such as through patternrecognition.

Example thermal image 700B, which can be taken one minute after spill306 occurred, illustrates that edge 706B appears slightly darker thanedge 706A, indicative at least in part of a lower relative temperature.Center spill area 708B also appears relative darker and more visuallydefined than center spill area 708A. A person having ordinary skill inthe art should appreciate that some deviation in temperaturemeasurements is possible due to measurement deviations, ephemeralphenomenon, instabilities in the measuring environment, noise, etc.Accordingly, the apparent slight uptick in temperature (e.g., asindicated in bar 710B and temperature display panel 702B as compared tobar 710A and temperature display panel 702A) may not represent an actualincrease in temperature.

Similarly, in example thermal image 700C, which can be taken two minutesafter spill 306 occurred, edge 706C and center spill area 708C appearrelatively darker as compared to edge 706B and center spill area 708B.Similarly, in thermal image 700D, which occurred three minutes afterspill 306 occurred, edge 706D and center spill area 708D appearrelatively darker as compared to edge 706C and center spill area 708C.And finally, in thermal image 700E, which occurred five minutes afterspill 306 occurred, edge 706E and center spill area 708E appearrelatively darker as compared to edge 706D and center spill area 708D.

Accordingly, as more time passes after spill 306 occurred, theappearance of spill 306 as imaged by an infrared camera of spilldetector 112 becomes more defined as compared to surface 604. Aspreviously mentioned, spill 306, which was imaged in thermal images700A-700E, was of a substantially similar temperature as surface 604when spill 306 occurred. Accordingly, spill 306, as imaged as spillimage 716A-716E, became more visible in the infrared camera as spill 306cooled by evaporative cooling. In some implementations of thisdisclosure, a thermal image of spill 306 can be taken. Based at least inpart on being able to distinguish spill 306 from surface 604 through thethermal image, spill 306 can be identified. For example, one or more ofspill image 716A-716E can be identified using image segmentation.

As previously mentioned, in some implementations, spill detector 112 isconnected to robot 100. Where robot 100 is mobile (such as where robot100 is a floor cleaner (e.g., floor scrubber), cart, or any other robotdescribed in this disclosure), robot 100 may be moving for periods oftime. Accordingly, robot 100 may not have minutes to wait for spill 306to become visible.

In some implementations, robot 100 can have temperature adjuster 116 (asdescribed with reference to FIG. 3 as well as elsewhere throughout thisdisclosure) to facilitate temperature change of spill 306 so that spill306 becomes more discernable in images, and more easily segmented.

For example, FIG. 8 illustrates an example fan 800 configured to blowair onto spill 306. Temperature adjuster 116 can include fan 800. Fan800 can include mechanical fans, fans with blades, bladeless fans,centrifugal fans, propeller fans, vanaxial fans, etc. Fan 800 can blowair directionally, such as blowing air in a direction of spill 306.

By blowing on spill 306, fan 800 can facilitate the lowering of thetemperature of spill 306, which can allow spill 306 to be morediscernable when imaged by an infrared camera of spill detector 112. Forexample, fan 800 can accelerate evaporative cooling of spill 306,causing spill 306 to cool faster. By way of illustration, thediscernibility as reflected in thermal image 700B-700E, which were takenminutes after spill 306 occurred, could be reflected in a thermal imagetaken seconds after spill 306 occurred with example fan 800 blowing ontospill 306.

When attached to robot 100, and as part of temperature adjuster 116, fan800 can be positioned distally facing from back side 124, and blowdistally from back side 124. Advantageously, this can allow fan 800 tofacilitate imaging of spills that originate from robot 100. For example,where robot 100 is a floor cleaning unit, such as a floor scrubber,spills can emanate from robot 100 in ways described herein withreference to FIG. 1 as well as elsewhere throughout this disclosure.When robot 100 moves forward, the spills (e.g., spill 306) can come intothe field of view 602 of spill detector 112. Advantageously, where spilldetector 112 is attached to a mobile robot 100, accelerating the coolingof spill 306 can allow spill detector 112 to detect spill 306 beforerobot 100 moves and spill 306 is out of range (e.g., out of field ofview 602) of spill detector 112. However, other placements of fan 800(and temperature adjuster 116) as well as spill detector 112 are alsocontemplated. Fan 800 and spill detector 112 can be positioned anywhereon the body of robot 100, such as on right side 126, front side 122,left side (not illustrated), underneath robot 100, on top of robot 100,etc. Advantageously, where robot 100 seeks out spills, having spilldetector 112 and/or temperature adjuster 116 extend in a forwarddirection from front side 122 can allow robot 100 to detect spills infront of it. Having spill detector 112 and/or temperature adjuster 116beneath robot 100 can allow robot 100 to detect spills robot 100 passesover. As mentioned in this disclosure, fan 800 and spill detector 112can be positioned elsewhere, not on the body of robot 100.

Other apparatuses can be used in the alternative or in combination withfan 800 in spill detector 112 in order to facilitate cooling of spill306 relative to surface 604. For example, a cool air stream can becreated using suction, such as by a combination of evaporative andadiabatic and/or Joule-Thomson cooling. Here, in some cases, a suctionhose can serve as a heat sink for the heat exchange due to, for example,the Joule-Thomson effect for cooling.

As another example, one or more temperature measurement devices (e.g.,thermocouples, thermistors, IR sensors, etc.) may be incorporated onrobot 100 to provide information (e.g., direct or indirect, absolute orrelative, etc.) about the temperature of the cleaning fluid, floor,exhaustion, parts of robot 100, and/or the environment. The data fromthese measurement devices can be used by spill detector 112, and methodsand/or algorithms performed by spill detector 112, to improve spilldetection performance.

As another example, the specific heat of spill 306 and surface 604 canbe different. Accordingly, cooling or heating surface 604 as well asspill 306 can enhance their contrast in an image, such as a thermalimage taking by a thermal camera (e.g., IR camera). FIG. 9 illustratesan example heating apparatus 900 that can heat spill 306. Temperatureadjuster 116 can include heating apparatus 900. In some implementations,heating apparatus 900 can include an electric heater, infrared heater,heat gun, furnace, water heater, oil heater, and/or any heater known inthe art. For example, and without limitation, where heating apparatus900 is an electric heater, it can comprise a fan and heating coil (e.g.,high resistance wires). The heating coils can heat proximal air, and thefan can blow the warmed air from heating apparatus 900.

Heating spill 306 and surface 604 can enhance imaging of spill 306 witha camera, such as an IR camera, because spill 306 can heat at adifferent rate than surface 604. FIG. 10A-10C illustrates thermal images1000A-1000C, where heating unit 900, appearing as heating unit images1020A-1020B, heats spill 306, appearing as spill images 1016A-1016C.Surface 604 appears as surface images 1018A-1018C. Substantially similarto thermal images 700A-700E described with reference to FIGS. 7A-7E,thermal images 1000A-1000C include reticules 1004A-1004C, temperaturedisplay panels 1002A-1002C, and bars 1010A-1010C.

Thermal image 1000A can be taken as heating unit 900 comes to bear.Accordingly, spill 306 and surface 604 have yet to be heated by heatingunit 900. As such spill image 1016A appears as a substantially similarcolor to surface image 1018A, and in some instances, may be difficult todistinguish and/or segment because of that substantial similarity.

Thermal image 1000B was taken when heating unit 900 directly hits aportion of spill 306, appearing as image portion 1022B. Image portion1022B illustrates the area where heat from heating unit 900 heats spill306 and surface 604. The portion of image portion 1022B corresponding tosurface image 1018B appears whiter as compared to the portion of imageportion 1022B corresponding to spill image 1016B. In some cases, thiscolor difference may be due to a temperature difference, wherein spill306, as represented by spill image 1016B, changes temperature moreslowly than surface 604, represented by surface image 1018B.Accordingly, when heat from heating unit 900 heats a portion of surface604 and spill 306, this portion represented by image portion 1022B, thetemperature of surface 604 heated by heating unit 900 heats faster thanthe portion of spill 306 heated by heating unit 900. This createstemperature differentiation, which can result in color differentiationin thermal image 1000B. Further temperature/color differentiation canfacilitate segmentation and allow spill 306 to be more readilyidentified.

Thermal image 1000C can be taken when heating unit 900 hits a largerportion of surface 604 and spill 306, such as by being further awayand/or having a wider spread (e.g., by adjusting airflow or the size ofthe aperture in which hot air flows). The results are substantiallysimilar to what was observed in thermal image 1000B. Image portion 1022Cillustrates the area where heat from heating unit 900 heats spill 306and surface 604. Again, the portion of image portion 1022C correspondingto surface image 1018C appears whiter as compared to the portion ofimage portion 1022C corresponding to spill image 1016C. Having suchdistinguishability can further facilitate segmentation and/or theidentification of spill 306.

Returning to FIG. 9, in some implementations, heating apparatus 900 canbe an exhaust of robot 100. Robot 100 may have exhausts, or otherapparatuses, to disperse heat from, for example, friction, motors,electronic parts, and/or any component of robot 100. In some cases, theexhaust can be an aperture that allows airflow, such as airflow overheat sinks. In some cases, the exhaust can be coupled with fans and/orother mechanisms for facilitated convection, such as pumps, suctiondevices, exchangers, radiators, etc.

As previously mentioned, temperature adjuster 116, and consequentlyheating apparatus 900, can be positioned distally facing from back side124. Advantageously, this can allow heating apparatus 900 to facilitateimaging of spills that originate from robot 100. For example, whererobot 100 is a floor cleaning unit, such as a floor scrubber, spills canemanate from robot 100. Thus, when robot 100 moves forward, the spills(e.g., spill 306) can come into field of view 602 of spill detector 112.Advantageously, where spill detector 112 is attached to a mobile robot100, heating spill 306 and surface 604 can allow spill detector 112 todetect spill 306 before robot 100 moves and spill 306 is out of range(e.g., out of field of view 602) of spill detector 112. However, otherplacements of heating apparatus 900 (and temperature adjuster 116) aswell as spill detector 112 are also contemplated. Heating apparatus 900and spill detector 112 can be positioned anywhere on the body of robot100, such as on right side 126, front side 122, left side (notillustrated), underneath robot 100, on top of robot 100, etc.Advantageously, where robot 100 seeks out spills, having spill detector112 and/or temperature adjuster 116 extend in a forward direction fromfront side 122 can allow robot 100 to detect spills in front of it.Having spill detector 112 and/or temperature adjuster 116 beneath robot100 can allow robot 100 to detect spills robot 100 passes over. Asmentioned in this disclosure, heating apparatus 900 and spill detector112 can be positioned elsewhere, not on the body of robot 100.

In some cases, a natural exhaust of robot 900 can be modified to directthe exhaust in a different direction, such as distally facing from backside 124. Such a modification can be made by appending tubing (e.g.,metal tubing) to a pre-existing exhaust to the desired direction (e.g.,distally facing from back side 124, right side 126, front side 122, leftside (not illustrated), underneath robot 100, on top of robot 100,etc.). In some cases, the appended tubing may be coupled to a valve toprovide a seal and/or to direct air flow from the pre-existing exhaustto the appended tubing. There can be other ways heating and/or coolingspill 306 and/or surface 604 to facilitate imaging. For example,microwaves, radiation, and/or electromagnetic waves can be emitted,which can causes, at least in part, greater temperature change in spill306 than in surface 604, causing spill 306 to be distinguishable fromsurface 604 in a thermal image.

In some implementations, the temperature of spill 306 (e.g., water,aqueous solutions, oil, etc.) itself can be changed to enhance imaging.For example, water, aqueous solutions, oil, etc. in robot 100 can beheated, such as by using a heater and/or by heat from, for example,friction, motors, electronic parts, and/or any component of robot 100.If the heat of the water, aqueous solutions, oil, etc. in robot 100exceeds that of the environment and/or surface 604, when the water,aqueous solutions, oil, etc. spill onto surface 604 as spill 306, spill306 will be hotter than surface 604. Accordingly, spill 306 and surface604 can be readily distinguished in a thermal image by camera 112.Similarly, in some implementations, water, aqueous solutions, oil, etc.in robot 100 can be cooled, such as with air conditioner(s),refrigerator(s), heat exchanger(s), fan(s), suction(s), cooling bead(s),etc. In some cases, such cooling can be accomplished through evaporativecooling, adiabatic expansion, Joule Thomson effects, and otherthermodynamic effects. If the water, aqueous solutions, oil, etc. inrobot 100 are cooler than the environment and/or surface 604, when thewater, aqueous solutions, oil, etc. spill onto surface 604 as spill 306,spill 306 will be cooler than surface 604. Accordingly, spill 306 andsurface 604 can be readily distinguished in a thermal image by camera112.

If spill 306 does not originate from robot 100, the water, aqueoussolutions, oil, etc. of spill 306 can be a different temperature basedat least in part on where it was before spilling. For example, in astore, certain items may be heated, such as under heat lamps, lights,heaters, etc. If these items spill as spill 306, spill 306 may be warmerthan surface 604 and can be readily distinguished in a thermal image bycamera 112. As another example, in a store, certain items may be cooled,such as by refrigeration, freezing, etc. If these items spill as spill306, spill 306 may be cooler than surface 604 and can be readilydistinguished in a thermal image by spill detector 112.

In some cases, spill detector 112 can image after a predetermined amountof time (e.g., 5, 10, 15, 20, 25, 30 or more seconds, or 1, 2, 3, 4, ormore minutes). The predetermined amount of time can depend on thedesired distinguishability of the spill 306 from surface 604, where thelonger the time, the more distinguishable spill 306 is from surface 604.Also, temperature adjuster 116 can make spill 306 more distinguishablefrom surface 604 in less time. However, this predetermined amount oftime can be weighed against a desire to cover more area for spilldetection, and other practical limitations such as a desire to clean afloor in a desired amount of time. For example, where spill detector 112is attached to robot 100, robot 100 can stop periodically (e.g., after apredetermined distance, such as 1, 2, 3, 4 or more feet, dependent onthe field of view of spill detector 112) to allow spill detector 112 todetect spills.

III. Other Sensors for Detecting Spills

In some implementations, additional information can be obtained inaddition to or in the alternative to image(s). For example, in someimplementations, spill detector 112 can include other sensors, such asany of the aforementioned sensors discussed with reflectance and/oremission to FIG. 3. Spill detector 112 can process the information fromthese other sensors along with any image(s) taken. In some cases, if oneor more of the image(s) and information from these other sensors areindicative at least in part of a spill 306, spill detector 112 (and/orrobot 100) can prompt a user to get feedback (e.g., using display 502)and/or perform an action in response to detecting a spill 306.

For example, spill detector 112 can include one or more light metersthat can detect the light reflectance and/or emission off surfaces. Insome cases, ambient light can reflect off spill 306 and/or surface 604.In some cases, light projected from spill detector 112, such as lightfrom a light bulb, light emitting diode (“LED”), lamp, laser, flash,and/or any other light source can be included in spill detector 112.From the ambient light and/or light projected from spill detector 112,spill 306 can have different reflectance and/or emission properties thansurface 604. These reflectance and/or emission properties include theamount (e.g., intensity) of light reflected off the surfaces, angle ofreflected light, spectrum of reflected light, amount of specular andLambertian reflectance, polarization (e.g., detection of which can befacilitated by projecting light from spill detector 112 at orsubstantially near Brewster's angle), light reflectance and/or emissionpatterns (e.g., reflectance and/or emission due to movement of spill 306due to, for example, vibrations (e.g., caused by robot 100 or otherenvironmental factors) and/or other perturbation(s) of the surface ofspill 306), and other properties. For example, a fan, a speaker (e.g., aloud speaker), and/or any other source of perturbation may be used tocreate patterns (e.g., ripples) on the surface of the spill, which couldthen be detected by the thermal infrared imager or other opticaldetection devices, as described in this disclosure. Laser, LED, or othersources of light (e.g., visibile or invisible) may be used to make thesaid patterns more detectable, or to make the patterns stand out morerelative to a dry floor surface.

In some cases, these reflectance and/or emission properties can bedependent on one another, such as, where the intensity of lightreflected off the surfaces is a function of angle and polarization oflight incidence to the surfaces. For example, spill 306 can have a firstreflectance and/or emission property and surface 604 can have a secondreflectance and/or emission property. Accordingly, spill detector 112can detect spill 306 based at least in part on the detection of thefirst reflectance and/or emission property by the one or more lightmeters. The different reflectance and/or emission properties ofdifferent materials (e.g., of surface 604 and spill 306) can be storedin memory 302, wherein controller 304 can identify spill 306 based atleast in part upon matching measured reflectance and/or properties toreflectance and/or emission properties stored in memory 302 and/or adifference between measured reflectance and/or emission properties indifferent spaces and/or different times. In some cases, at leastdetecting different reflectance and/or emission properties can promptspill detector 112 to detect a spill 306 and/or alert a user usingdisplay 502 (illustrated in FIG. 5), wherein the user can view panel 506and determine if spill 306 has been detected. In some cases, spilldetector 112 can process the information from the one or more lightmeters along with any image(s) and/or information from other sensor. Insome cases, spill detector 112 can prompt a user to get feedback (e.g.,using display 502) and/or perform an action (e.g., a stop, alert help,or ignore) in response to detecting a spill 306 in response to at leastfinding a difference in reflectance and/or emission.

As another example, in some implementations, spill detector 112 caninclude a pad extending from robot 100, the pad can be in contact withthe floor (e.g., surface 604). For example, the pad can extend distallyfrom back side 124 and/or distally from any other side of robot 100. Forexample, where the pad extends distally from back side 124, it can morereadily detect spills from robot 100. Where pad extends from front side122 can allow robot 100 to detect spills in front of it. Having the padbeneath robot 100 can allow robot 100 to detect spills it passes over.The pad can be attached to a pole and/or any structure configured toposition the pad in contact with the floor/surface. Robot 100 and/orspill detector 112 can use the resistance of the pad as it moves on thefloor to take measurements indicative at least in part of the friction(and/or the coefficient of friction) and/or slip resistance between thepad and surface 604 and/or spill 306. In some cases, the coefficient offriction of surface 604 can be different than spill 306. For example,spill 306 can comprise liquids and/or other substances (e.g., water,aqueous solutions, honey, milk, mustard, ketchup, oil, bodily fluids,beverages, butter, ice, candy, cleaners (e.g., cleaning fluid, floorwax, disinfectants, etc.), grease, oil, industrial waste, coolant,and/or other chemicals) with lower coefficients of friction than thematerials of surface 604 (e.g., wood (e.g., engineered, synthetic,hardwood, etc.), bamboo, vinyl, concrete, ceramic tile, linoleum,porcelain tile, laminate, cork, stone, aluminum, metals, steel, epoxy,and/or other materials). In some cases, in order to measure the frictionexperienced by the pad, a dynamometer can be used. In some cases, thedynamometer can be coupled to a motor of robot 100 (e.g., a motor ofactuators unit 320, such as the motor used for the motorized propulsionthat enables robot 100 to move from one place to another) to detectresistance of movement (e.g., as experienced in differences in torque,power, force, etc.). In some cases, dynamometer and/or other kinematicmeasurements (e.g., accelerometers, gyroscopes, etc.) can be coupled tothe pad in order to detect increased resistance of movement (e.g., asexperienced by moments, forces, etc.) on the pad. In some cases, inresponse to at least detecting changes in the friction experienced bythe pad, spill detector 112 can detect a spill 306 and/or prompt spilldetector 112 to alert a user using display 502, wherein the user canview panel 506 and determine if spill 306 has been detected. In somecases, spill detector 112 can process the information from thedynamometer along with any image(s) and/or information from othersensor. In some cases, spill detector 112 can prompt a user to getfeedback (e.g., using display 502) and/or perform an action (e.g., astop, alert help, or ignore) in response to detecting a spill 306 inresponse to at least finding a difference in friction.

As another example, spill detector 112 can include a fluorescencedetector and/or imager. For example, there can be a fluorescent waxadditive for the floor (e.g., surface 604). Some example additives arequinine, niacin, riboflavin, vitamins A and B, chlorophyll, bleach,fluorescent whitener additives, uranin, metal complex organic pigments,aromatic organic pigments, and/or other known fluorescent chemicals,such as pigments and/or dyes. Under a blacklight UV source (or otherfluorescent-inducing conditions, such as laser-induced fluorescence,etc.), which can be attached to robot 100 or elsewhere in theenvironment such that the blacklight UV source shines on the floor(e.g., surface 604), spill 306 may attenuate or occlude the incident UVand/or the UV-induced floor fluorescence, which can be detected bypotentially sharp boundary drops in the fluorescence detected by thefluorescence detector and/or imager. Accordingly, spill detector 112 candetect spill 306 based at least in part on the boundaries of data (e.g.,an image) by a fluorescence detector and/or imager. As anotherillustration, where robot 100 is a floor cleaner (e.g., floor scrubber),the cleaning fluid of robot 100 can have a UV-induced fluorescence. ThisUV-induced fluorescence can be mitigated once the cleaning fluid dries.In this way, where spill 306 comprises such cleaning fluid, spill 306can be imaged (e.g., using an RGB camera or other camera) anddistinguishable from surface 604 under a black light UV source or otherfluorescent-inducing conditions. In some cases, spill detector 112 canprocess images and detect the UV-induced fluorescence of spill 306, suchas through machine vision algorithms including machine learning, imageprocessing, segmentation, etc. In some cases, at least detecting theUV-induced fluorescence in some cases (or in some cases, the occlusionof UV-induced fluorescence) can prompt spill detector 112 to detect aspill 306 and/or alert a user using display 502, wherein the user canview panel 506 and determine if spill 306 has been detected. In somecases, spill detector 112 can process the information from the image(s)imaged under a black light UV source and/or other fluorescent-inducingconditions along with any other image(s) and/or information from othersensor. In some cases, spill detector 112 can prompt a user to getfeedback (e.g., using display 502) and/or perform an action (e.g., astop, alert help, or ignore) in response to detecting a spill 306 inresponse to at least detecting UV-induced fluorescence in an image takenunder a black light UV source or other fluorescent-inducing conditions.

As another example, luminescent components (e.g., chemicallyluminescent) may be added to cleaning fluids, such as by mixing in withthe detergent, water, and/or dispensed from a separate container and/orreservoir. Luminescence may be induced by chemical reaction betweenthose components, contact with air, fluid agitation during cleaningaction, friction, etc. Likewise, a chemical reaction or a physicalchange in a state of matter can be used to alter the temperature of thecleaning fluid to help with spill detection, separately and/or incombination with systems and methods described in this disclosure. Forexample, latent heat of solution or melting can be utilized to alter thetemperature of the cleaning medium.

As another example, an additional absorbent material pad may beemployed, wherein the capacitance or other properties of the pad wouldchange as it absorbs spills. In some cases, the pad can extend fromfront side 122 and/or back side 124.

As another example, spill detector 112 can include a camera (e.g., photocamera, video camera, IR camera, etc.) that can image surface 604 andspill 306. Spill detector 112 can use the camera to detect a movement ofspill 306 on surface 604. For example, spill 306 can expand and/or movedue to surface 604 being unlevel and/or uneven, and/or due to propertiesof spill 306, such as adhesion, cohesion, etc. In some cases, the cameracan take a plurality of images of surface 604 and spill 306. Adifferential between the images can be taken, which, in some cases, canproduce a differential image. The differential image can be used todetermine, at least in part, where (and if) changes have occurred insurface 604 and spill 306. If the changes indicate at least in part amovement, spill detector 112 can detect spill 306 based at least in parton that movement. In some cases, at least detecting the movement canprompt spill detector 112 to detect a spill 306 and/or alert a userusing display 502, wherein the user can view panel 506 and determine ifspill 306 has been detected. In some cases, spill detector 112 canprocess the detected movement along with any other information fromother sensor. In some cases, spill detector 112 can prompt a user to getfeedback (e.g., using display 502) and/or perform an action (e.g., astop, alert help, or ignore) in response to detecting a spill 306 inresponse to at least detecting a movement.

As another example, spill detector 112 can measure electric propertiesof surface 604 and spill 306. By way of illustration, spill detector 112can have a plurality of electrodes and/or leads. In some cases, theelectrodes and/or leads can make contact with surface 604 and/or spill306. For example, spill detector 112 can include a capacitance meter,voltmeter, multimeter (e.g., a Digital Multimeter (“DMM”)),oscilloscope, ohmmeter, ammeter, etc. In some cases, surface 604 andspill 306 can have different electrical properties, where spill detector112 can use, at least in part, the different electrical properties ofsurface 604 and spill 306 to detect spill 306. For example, in somecases where spill 306 is an aqueous solution, spill 306 can have greaterconductivity, less impedance, and/or greater capacitance than floor 306when surface 604 includes materials, such as wood (e.g., engineered,synthetic, hardwood, etc.), bamboo, vinyl, concrete, ceramic tile,linoleum, porcelain tile, laminate, cork, stone, epoxy, and othermaterials. In other cases, spill 306 can have less conductivity, moreimpedance, and/or less capacitance than surface 604 when surface 604comprises, for example, certain metals. A person having ordinary skillin the art should appreciate that spill 306 and surface 604 can comprisevarious materials, each of which with different relative electricalproperties. Based at least in part on the difference in electricalproperties, spill detector 112 can detect spill 306. For example,controller 304 can receive electrical measurements from spill detector112. Controller 304 can detect a difference in electrical properties,such as a difference in conductivity, impedance, and/or capacitance. Insome cases, the electrical properties of surface 604 can bepredetermined for controller 304, such as by programming and/oridentification of the materials. Accordingly, based at least on thepredetermined electrical properties of surface 604, a higher or lowermeasurement of electrical properties can be indicative of spill 306. Insome implementations, at least any difference (e.g., higher in somecircumstances, lower in some circumstances, or either higher or lower insome circumstances) of electrical properties detected can promptcontroller 304 to alert a user and/or use other sensors of spilldetector 112 to verify the presence of spill 306. In some cases, atleast detecting a difference in electrical properties can prompt spilldetector 112 to detect a spill 306 and/or alert a user using display502, wherein the user can view panel 506 and determine if spill 306 hasbeen detected. In some cases, spill detector 112 can process thedetected electrical property difference along with any image(s) and/orother information from other sensor. In some cases, spill detector 112can prompt a user to get feedback (e.g., using display 502) and/orperform an action (e.g., a stop, alert help, or ignore) in response todetecting a spill 306 in response to at least finding a difference inelectrical properties.

In some implementations, spill detector 112 can include a microphone.The microphone can be configured to detect noises associated with aspill, such as the sound of broken glass, objects falling, people'sreactions, etc. These noises associated with a spill can be stored inmemory 302, wherein controller 304 can associate the noises with spill306. In some cases, at least the detection of the noises can promptspill detector 112 to detect a spill 306 and/or alert a user usingdisplay 502, wherein the user can view panel 506 and determine if spill306 has been detected. In some cases, spill detector 112 can processdetected noises along with any image(s) and/or other information fromother sensor. In some cases, spill detector 112 can prompt a user to getfeedback (e.g., using display 502) and/or perform an action (e.g., astop, alert help, or ignore) in response to detecting a spill 306 inresponse to at least detecting the noises.

In some cases, spills can be associated with colors. For example,certain cleaning products and/or other chemicals can have an associatedcolor or be dyed a color. Where there is a spill 306, the color of thatcleaning product and/or other chemical can be more readily viewable thanif the cleaning products and/or other chemicals are spread out, such asby cleaning and drying. A camera (e.g., an RGB camera) of spill detector112 can be used to image spill 306 and surface 604. Based at least onthe colors, spill detector 112 can segment the image and/ordetect/identify spill 306. In some cases, based at least upon thedetection/identification of spill 306 from the images, spill detector112 can alert a user using display 502, wherein the user can view panel506 and determine if spill 306 has been detected. In some cases, spilldetector 112 can process these images along with any other image(s)and/or other information from other sensor. In some cases, spilldetector 112 can prompt a user to get feedback (e.g., using display 502)and/or perform an action (e.g., a stop, alert help, or ignore) inresponse to detecting a spill 306 in response to at least detecting thenoises.

In some implementations, location can be used to further informmeasurements taken by spill detector 112. For example, where spilldetector 112 is attached to robot 100, robot 100 can have a mapping andlocalizing unit that allows robot 100 to determine its location in anenvironment. As another example, where spill detector 112 is stationary,it can associate the location spill detector 112 with characteristics.

In some implementations, spill detector 112 can then learn to associatereadings (e.g., images taken by cameras and/or any other information ofany other sensor of spill detector 112) with locations in theenvironment. For example, in a store environment, some areas can becolder due to refrigeration, air conditioning vents, and/or other storefeatures. Some areas can also be warmer due to heating vents, heatexchanges, and/or other store features. In some implementations, spilldetector 112 can learn to associate certain temperature readings withcertain locations in an environment. For example, these temperaturereadings associated with certain locations can facilitate calibration ofIR cameras and/or any other sensor of spill detector 112. Moreover, ifthe temperature of the floor (e.g., surface 604) is known, spilldetector 112 can identify differences from that temperature as areas ofpotential spills 306.

In some implementations, through a plurality of iterations where spilldetector 112 detects a potential spill and alerts a user using display502, the user can provide feedback regarding the spill detection (e.g.,with regard to the veracity or legitimacy of the detection). In somecases, where the user selects an action such as option 512, whichignores the spill, spill detector 112 can learn to associate imagestaken at particular locations (e.g., as determined by a mapping andlocalizing unit of robot 100) with false positives. Accordingly, forexample, spill detector 112 can recalibrate the cameras (e.g., IRcamera) and/or other sensors in those locations and/or increasepredetermined thresholds to decrease the number of false positives. Forexample, the predetermined number threshold and/or the predeterminedpercentage threshold discussed with reference to at least portion 1106of FIG. 11 can be adjusted. As another example, the other sensors ofspill detector 112 can also be recalibrated to decrease false positives.

In some cases, spill detector 112 can learn to associate certain areasas having an increased likelihood of spills. For example, through aplurality of iterations, spill detector 112 can detect a potential spilland receive confirmation (e.g., an acknowledgment or any action such asactions associated at least in part with options 508, 510) that a spillhas been detected. If the detected spills frequently appear in aparticular area (e.g., as determined by the mapping and localizing unitof robot 100), spill detector 112 can more readily identify potential asspills, such as by recalibrating a camera (e.g., an IR camera) in thoselocations and/or decreasing predetermined thresholds (e.g., thepredetermined number threshold and/or the predetermined percentagethreshold discussed with reference to at least portion 1106 of FIG. 11).As another example, the other sensors of spill detector 112 can also berecalibrated in order to more readily detect spills (e.g., spill 306) inthese areas. Advantageously, using locations can further enhance therobustness and capabilities of spill detector 112's ability to detectspills and reduce false positives.

IV. Methods

FIG. 11 illustrates a process flow diagram of an example method fordetecting a spill 306. Portion 1102 can include obtaining image(s) of ascene containing a potential spill. An example image can be an image ofat least a portion of field of view 604. For example, these image(s) caninclude thermal images, such as the thermal images discussed withreference to the “II. Thermal imaging to detect spills” section of thisdisclosure, as well as elsewhere throughout this disclosure. Thedistinction between spill 306 and surface 604 in those thermal imagescan be enhanced by one or more of the systems and methods described inthat section such as by using temperature adjuster 116. Any other imagedescribed in this disclosure can also be used as well.

Additional sensors can also be used in addition to or in the alternativeto the images. For example, any of the sensors discussed in the “III.Other sensors to detect spills” section of this disclosure, as well aswell as elsewhere throughout this disclosure, can also be used asadditional sensors.

In some cases, spill detector 112 can remove noise from image(s)obtained in portion 1102. In some cases, infrared cameras can be noisy,which can impair the ability of spill detector 112 to image and/ordetect spill 306. Accordingly, filters can be applied to the image(s) inorder to remove noise. These filters can filter full field and/orspecular components of images (e.g., thermal images). In some cases, alow pass filter can be used to filter out ambient noise. In some cases,high pass filters can be used to remove aliasing and/or other noise. Insome cases, a bandpass filter and/or a combination of high pass filtersand low pass filters can be used to retain the informative spectrum ofthe images. In some cases, these filters can be applied in multiplestages. For example, in some cases with thermal images, the bandpass canbe approximately wavelengths 2.7-5.3 μm, 8-12 μm, and/or any otherwavelengths determined based at least in part on infrared cameracalibration, the expected temperatures of spill 306, the expectedtemperature of surface 604, the expected temperature of theenvironment/scene, predetermined environmental/scenic noise, the rangeof temperatures in the environment/scene etc. Other noise-reducingsignal processing can be used, such as removing outliers, correlation,averaging, etc.

In some cases, other environmental/scenic noise can impair the qualityof the image(s) obtained in portion 1102. For example, heat signals fromthe robot 100 itself can problematically skew images and/or decrease theresolution of images. To fix this issue, in some cases, a hood and/or athermal shield can be used on the body of robot 100 to reduce robot's100 appearance in images. In some cases, images can also be cropped toremove the appearance of robot 100. False positives can also beincreased by other factors, such as lighting, vents, etc. A hood can beplaced over spill detector 112 to reduce that noise.

Portion 1104 includes identifying the potential spill (e.g., spill 306).In some implementations, spill detector 112 can be configured to performsegmentation on image(s) obtained from portion 1102. There are manyknown image segmentation techniques known in the art, and thisdisclosure is not limited to any particular set of them. For example,such segmentation can include thresholding, water shed techniques,clustering, neural networks, K-means, region-growing, edge detection,etc. By way of illustration, adaptive thresholding can segment theimage(s) taken from portion 1102. Advantageously, segmentation can allowspill detector 112 to determine which portions of an image belong to,for example, spill 306 or surface 604. The images, after and/or beforesegmentation, can also be further cleaned up with morphological imageprocessing (e.g., erosion and dilation to remove noise), filters, etc.

As discussed in the “II. Thermal imaging to detect spills” section,where thermal images are used, the distinction between spill 306 andsurface 604 can be resolvable. As discussed, this distinction can befurther enhanced in images over time and/or with the use of temperatureadjuster 116. These distinctions can further enable effectivesegmentation, such as by enhancing the edges and/or boundaries of spill306, making those edges more easily detected by edge detection methodsand/or other segmentation. These distinctions can also cause thetemperature difference between spill 306 and surface 604 to be greater.As a result, they appear as more dissimilar luminance values,chrominance, and/or colors in an image (e.g., thermal image), allowingfor thresholding and/or other segmentation techniques to be moreeffective due to the ability to choose thresholds that can separatespill 306 and surface 604. In this way, discussed systems and methods inthe “II. Thermal imaging to detect spills” section can reduce falsepositives and enable spill detector 112 to better segment and imageand/or identify spill 306.

In some implementations, a vision classifier can be used on the image(s)obtained in portion 1102 to identify the potential spills. In somecases, the vision classifier can utilize learning-based methods toidentify spills from image(s) obtained in portion 1102. By way ofillustrative example, library 324 can comprise example images of spills,such as example thermal images, RGB camera images, etc., of spills.Library 324 can then be used in a supervised or unsupervised machinelearning algorithm for controller 304 to learn to identify/associatepatterns in images with spills. The images of library 324 can beidentified (e.g., labelled by a user (e.g., hand-labelled) orautomatically, such as with a computer program that is configured togenerate/simulate library images of spills and/or label those libraryimages). In some implementations, library 324 can also include images ofspills in different lighting conditions, angles, sizes (e.g.,distances), clarity (e.g., blurred, obstructed/occluded, partially offframe, etc.), colors, temperatures, surroundings, etc. From theseimages, controller 304 can first be trained to identify the spills.Spill detector 112 can then use that training to identify spills inimage(s) obtained in portion 1102.

For example, in some implementations, controller 304 can be trained fromlibrary 324 to identify patterns in library images and associate thosepatterns to spills. When an image obtained in portion 1102 has thepatterns that controller 304 identified and associated to spills,controller 304 can determine that the image obtained in portion 1102contains a spill and/or the location of the spill in the image obtainedin portion 1102. In some implementations, controller 304 can processeach image obtained in portion 1102 and compare that image to one ormore images in library 324 (e.g., a library image). In some cases, wherean image obtained in portion 1102 substantially matches an image orplurality of images in library 324, controller 304 can identify theimage obtained in portion 1102 as containing a spill (e.g., spill 306)and/or the location of the spill in that image.

Portion 1106 includes determining a confidence that the spill has beendetected. In some implementations, the confidence can be calculatedusing, at least in part, Bayesian and/or other statistical methods. Insome implementations, the confidence can be determined by the number oftimes a spill is identified in the images obtained in portion 1102. Forexample, the images obtained in portion 1102 can comprise a plurality ofimages taken in succession. For example, spill detector 112 can takeimages in sub-second intervals. In some cases, these images can form avideo, wherein a camera of spill detector 112 takes images formingsubstantially a video, such as a video having 24 or more frames persecond. When spill detector 112 is stationary, spill detector 112 maytake many images of a substantially similar position, allowing spilldetector 112 to image a potential spill over a period of time. Whenspill detector 112 is mobile, such as attached to a mobile robot 100,spill detector 112 may image a potential spill for a period of time asit moves, wherein the potential spill eventually moves off frame. Ineither case, there can be a plurality of images of a potential spillobtained in portion 1102.

In some implementations, there can be a predetermined number thresholdwherein if equal to or more than the predetermined number threshold ofimages obtained in portion 1102 are identified as having spill 306,spill detector 112 determines that there is a spill with highconfidence. For example, the predetermined number threshold can be anumber (e.g., 1, 2, 3, 4, 5, 6, or more) wherein the predeterminednumber threshold can be determined from at least one or more of thespeed at which spill detector 112 (and/or robot 100) is moving, the sizeof field of view 602 of spill detector 112, the time in which apotential spill may be within field of view 602, whether temperatureadjuster 116 is in use, characteristics of temperature adjuster 116 tocreate differentiation between spill 306 and surface 604, the number ofimages taken per second, the kind of images taken (e.g., thermal, RGB,etc.), a percentage of the number of images taken per second, and/orother characteristics of spill detector 112, potential spill 306, and/orsurface 604. In some cases, the predetermined number threshold can bebased at least in part on images in a predetermined time interval (e.g.,1, 2, 3, 4 or more seconds). The predetermined number threshold can bedetermined based at least in part on the number of images taken persecond, the kind of images taken (e.g., thermal, RGB, etc.), apercentage of the number of images taken per second, and/or othercharacteristics of spill detector 112, potential spill 306, and/orsurface 604. In some cases, instead of, or in addition to, apredetermined number threshold, a relative measurement, such as apredetermined percentage threshold can be used, wherein when thepercentage of images in the predetermined time interval is equal to orgreater than the predetermined percentage threshold, spill detector 112determines that there is a high probability a spill has been detected.

Where spill detector 112 detects spills in some images obtained inportion 1102, but the number and/or percentage of images does not exceedthe predetermined number threshold and/or the predetermined percentagethreshold, spill detector 112 does not detect a spill with highconfidence. In some cases, when robot does not detect a spill with highconfidence, it is said to detect a spill with low confidence. Othersimilar thresholds can be used to stratify detection, such aspredetermined number thresholds and predetermined percentage thresholdsset to identify medium confidence, medium-to-low confidence,medium-to-high confidence, very high confidence, and/or any othercategorization indicative at least in part of confidence.

In some implementations, additional sensors can be used to furtherinform spill detectors confidence. For example, any sensor discussed inthe “III. Other sensors to detect spills” section of this disclosure, aswell as well as elsewhere throughout this disclosure, can provideadditional information. In some implementations, if spill detector 112does not determine with high confidence (e.g., determines with lowconfidence) a spill has been detected based at least in part on theimages obtained in portion 1102, spill detector 112 can determine withhigh confidence a spill has been detected if one or more of theseadditional sensors have information indicative at least in part of aspill. In some cases, where spill detector 112 detects a spill with highconfidence based at least in part on images obtained from portion 1102,that confidence may be reduced to low confidence (or another confidence)if the additional sensors do not detect a spill. However, in someimplementations, even if the additional sensors do not detect a spill,spill detector 112 can determine with high confidence that a spill hasbeen detected if the number and/or percentage of images obtained fromportion 1102 exceeds the predetermined number threshold and/or thepredetermined percentage threshold.

Portion 1108 includes determining if an action should be performed basedat least in part on the confidence. In some implementations, if a lowconfidence or a high confidence was determined in portion 1106, spilldetector 112 can alert a user using display 502 (described in FIG. 5),wherein the user can view panel 506 and determine if spill 306 has beendetected. The user can then use display 502 to perform an action, suchas an action associated at least in part with one or more of options508, 510, 512. Advantageously, in cases of low confidence, thisadditional feedback can allow spill detector 112 to determine that aspill has been detected and perform actions according to userinstruction. In cases where there is high confidence, a user can thenchoose the appropriate actions.

In some implementations, spill detector 112 (and/or robot 100) mayautomatically perform an action, such as one or more actions associatedwith options 508, 510, 512. In some cases, spill detector 112 can send asignal to robot 100 indicative at least in part of the action. Inresponse to the signal, robot 100 may then actuate one or more ofactuators 320. In some cases, through repeated selection by a user,robot 100 can learn to associate detected spills (e.g., based at leastin part on the confidence of portion 1106, patterns in images obtainedin portion 1102, and/or user inputs) with performed actions. In thisway, spill detector 112 can then perform those actions with little to nouser input. The user's input can also inform the other portions 1102,1104, 1106. For example, where a user selects option 512 to ignore adetected spill, through successive iterations, spill detector 112 canassociate the patterns of images from portion 1102 and/or informationfrom other sensors with being an ignored spill and/or false positive.Accordingly, spill detector 112 may no longer detect such patterns asbeing associated with spills.

FIG. 12 illustrates a process flow diagram of an example method fordetecting a spill 306 where robot 100 can ask for user assistance inidentifying a spill. In method 1200, portions 1202, 1204, 1206 can besubstantially similar to portions 1102, 1104, 1106 illustrated in FIG.11, respectively.

Portion 1208 can include determining if the spill detection meetsuncertainty criteria. For example, a low confidence can cause the spilldetection to meet the uncertainty criteria, wherein spill detector 112cannot determine the presence or absence of a spill sufficiently (e.g.,without substantial risk of false positives or false negatives). In someimplementations a confidence threshold can be used, wherein if theconfidence is below the confidence threshold, spill detector 112 candetermine that the uncertainty criteria has been met. The confidencethreshold can be determined at least in part on empirical data on falsepositives and false negatives, the resolution of sensors of spilldetector 112, the number of detection methods (e.g., how many of thespill detection methods described in this disclosure are used), and/orany other criteria. If the spill detection does not meet the uncertaintycriteria (e.g., spill detector 112 was sufficiently confident in thedetection), in portion 1210, spill detector 112 can return thedetermination of a spill or non-spill.

Portion 1212 can include asking for assistance. Spill detector 112 canask for assistance via user interface 318, server 400, and/or any othermedium. Asking for assistance can include sending a communication, suchas a message, alert, etc.

Portion 1214 can include receiving input identifying the spill. Theinput can be user input that is inputted via user interface 318, server400, and/or any other medium. The user input can include identificationof a spill, the location of a spill, and/or any other informationinputted by the user.

Portion 1216 can include learning from the input in portion 1214. Forexample, the user input can provide another labeled example for spilldetector 112 and/or robot 100 to input into library 324 and/or use withmachine learning. The labeled example can then be used to identifywhether a spill is present in other cases like it, such as by comparingthat labeled example to captured data and/or using learning algorithmsto associate patterns in the labeled example with captured data.

FIG. 13 illustrates a process flow diagram of an example method fordetecting a spill 306 where robot 100 can adjust behaviors based onfeedback. In method 1300, portions 1302, 1304, 1306, 1308 can besubstantially similar to portions 1102, 1104, 1106, 1108 illustrated inFIG. 11, respectively.

Portion 1310 can include receiving feedback on whether the action (e.g.,the action determined in portion 1308) was correct and/or ifidentification of the spill (e.g., the determination from portion 1304)was correct. The feedback can be inputted via user interface 318, server400, and/or any other medium. The feedback can include identification ofa spill, the location of a spill, confirmation/rejection of spilldetection, confirmation/rejection of actions, and/or any otherinformation inputted by the user.

Portion 1312 can include adjusting actions and/or spill determinationsbased at least in part on feedback received in portion 1310. Forexample, through repeated selection by a user, robot 100 can learn toassociate detected spills (e.g., based at least in part on theconfidence of portion 1306, patterns in images obtained in portion 1302,and/or user inputs) with performed actions. In this way, spill detector112 can then perform those actions with little to no user input. Theuser's input can also inform the other portions 1302, 1304, 1306. Forexample, where a user selects option 512 to ignore a detected spill,through successive iterations, spill detector 112 can associate thepatterns of images from portion 1302 and/or information from othersensors with being an ignored spill and/or false positive. Accordingly,spill detector 112 may no longer detect such patterns as beingassociated with spills.

FIG. 14 illustrates a process flow diagram of an exemplary method fordetecting spills in accordance with principles of the presentdisclosure. In method 1400, portion 1402 can include generating a firstimage of a first scene at a first location that contains a spill.Portion 1404 can include generating a second image of a second scene ata second location that contains no spill. Portion 1406 can includesegmenting the first image to detect the spill from at least thermalvalues in a segment of the first image. Portion 1408 can includeidentifying the spill. Portion 1410 can include generating an alertindicative at least in part of the identification of the spill.

As used herein, computer and/or computing device can include, but arenot limited to, personal computers (“PCs”) and minicomputers, whetherdesktop, laptop, or otherwise, mainframe computers, workstations,servers, personal digital assistants (“PDAs”), handheld computers,embedded computers, programmable logic devices, personal communicators,tablet computers, mobile devices, portable navigation aids, J2MEequipped devices, cellular telephones, smart phones, personal integratedcommunication or entertainment devices, and/or any other device capableof executing a set of instructions and processing an incoming datasignal.

As used herein, computer program and/or software can include anysequence or human or machine cognizable steps which perform a function.Such computer program and/or software may be rendered in any programminglanguage or environment including, for example, C/C++, C#, Fortran,COBOL, MATLAB™, PASCAL, Python, assembly language, markup languages(e.g., HTML, SGML, XML, VoXML), and the like, as well as object-orientedenvironments such as the Common Object Request Broker Architecture(“CORBA”), JAVA™ (including J2ME, Java Beans, etc.), Binary RuntimeEnvironment (e.g., BREW), and the like.

As used herein, connection, link, transmission channel, delay line,and/or wireless can include a causal link between any two or moreentities (whether physical or logical/virtual), which enablesinformation exchange between the entities.

It will be recognized that while certain aspects of the disclosure aredescribed in terms of a specific sequence of steps of a method, thesedescriptions are only illustrative of the broader methods of thedisclosure, and may be modified as required by the particularapplication. Certain steps may be rendered unnecessary or optional undercertain circumstances. Additionally, certain steps or functionality maybe added to the disclosed implementations, or the order of performanceof two or more steps permuted. All such variations are considered to beencompassed within the disclosure disclosed and claimed herein.

While the above detailed description has shown, described, and pointedout novel features of the disclosure as applied to variousimplementations, it will be understood that various omissions,substitutions, and changes in the form and details of the device orprocess illustrated may be made by those skilled in the art withoutdeparting from the disclosure. The foregoing description is of the bestmode presently contemplated of carrying out the disclosure. Thisdescription is in no way meant to be limiting, but rather should betaken as illustrative of the general principles of the disclosure. Thescope of the disclosure should be determined with reference to theclaims.

While the disclosure has been illustrated and described in detail in thedrawings and foregoing description, such illustration and descriptionare to be considered illustrative or exemplary and not restrictive. Thedisclosure is not limited to the disclosed embodiments. Variations tothe disclosed embodiments can be understood and effected by thoseskilled in the art in practicing the claimed disclosure, from a study ofthe drawings, the disclosure and the appended claims.

It should be noted that the use of particular terminology whendescribing certain features or aspects of the disclosure should not betaken to imply that the terminology is being re-defined herein to berestricted to include any specific characteristics of the features oraspects of the disclosure with which that terminology is associated.Terms and phrases used in this application, and variations thereof,especially in the appended claims, unless otherwise expressly stated,should be construed as open ended as opposed to limiting. As examples ofthe foregoing, the term “including” should be read to mean “including,without limitation,” “including but not limited to,” or the like; theterm “comprising” as used herein is synonymous with “including,”“containing,” or “characterized by,” and is inclusive or open-ended anddoes not exclude additional, unrecited elements or method steps; theterm “having” should be interpreted as “having at least;” the term “suchas” should be interpreted as “such as, without limitation;” the term‘includes” should be interpreted as “includes but is not limited to;”the term “example” is used to provide exemplary instances of the item indiscussion, not an exhaustive or limiting list thereof, and should beinterpreted as “example, but without limitation;” adjectives such as“known,” “normal,” “standard,” and terms of similar meaning should notbe construed as limiting the item described to a given time period or toan item available as of a given time, but instead should be read toencompass known, normal, or standard technologies that may be availableor known now or at any time in the future; and use of terms like“preferably,” “preferred,” “desired,” or “desirable,” and words ofsimilar meaning should not be understood as implying that certainfeatures are critical, essential, or even important to the structure orfunction of the present disclosure, but instead as merely intended tohighlight alternative or additional features that may or may not beutilized in a particular embodiment. Likewise, a group of items linkedwith the conjunction “and” should not be read as requiring that each andevery one of those items be present in the grouping, but rather shouldbe read as “and/or” unless expressly stated otherwise. Similarly, agroup of items linked with the conjunction “or” should not be read asrequiring mutual exclusivity among that group, but rather should be readas “and/or” unless expressly stated otherwise. The terms “about” or“approximate” and the like are synonymous and are used to indicate thatthe value modified by the term has an understood range associated withit, where the range can be ±20%, ±15%, ±10%, ±5%, or ±1%. The term“substantially” is used to indicate that a result (e.g., measurementvalue) is close to a targeted value, where close can mean, for example,the result is within 80% of the value, within 90% of the value, within95% of the value, or within 99% of the value. Also, as used herein“defined” or “determined” can include “predefined” or “predetermined”and/or otherwise determined values, conditions, thresholds,measurements, and the like.

What is claimed is:
 1. A robot comprising: an actuator configured tomove the robot between a plurality of locations; a spill detectorcomprising at least one optical imaging device configured to capture aplurality of images of a scene containing a spill while the robot movesbetween the plurality of locations; and a processor apparatus configuredto identify the spill in the plurality of images, and generate an alertindicative at least in part of a recognition of the spill; wherein: theprocessor is further configured to determine a confidence measure in theidentification of the spill; the generation of the alert indicative atleast in part of the recognition of the spill is based on the confidencemeasure meeting or exceeding a predetermined threshold, the confidencemeasure being determined based at least on a quantity of imagescontaining at least a portion of the identified spill relative to aquantity of images captured over a predetermined time interval; and thegeneration of the alert comprises a presentation of a plurality ofuser-selectable options associated with actions of the robot withrespect to the identified spill.
 2. The robot of claim 1, wherein the atleast one optical imaging device comprises an infrared camera and theplurality of images comprises thermal images.
 3. The robot of claim 1,further comprising a temperature adjuster configured to change atemperature value of the scene containing the spill.
 4. The robot ofclaim 3, wherein the temperature adjuster comprises at least one of anexhaust and a fan.
 5. The robot of claim 1, wherein the robot furthercomprises a sensor configured to detect at least one of reflectanceproperties, emission properties, electrical properties, noises, andfriction of the scene.
 6. The robot of claim 5, wherein the confidencemeasure is based at least in part on information from both the sensorand the at least one image.
 7. The robot of claim 1, wherein theprocessor is further configured to generate a color image having aplurality of colors based at least in part on a plurality of thermalvalues of a segment of the plurality of images, and the plurality ofcolors are indicative at least in part of the spill.
 8. The robot ofclaim 1, wherein the robot further comprises a floor cleaning system. 9.The robot of claim 8, wherein the processor is further configured toreceive an action command in response to the alert and to turn off thefloor cleaning system based at least in part on the received actioncommand.
 10. The robot of claim 1, wherein the processor is furtherconfigured to utilize a learning-based visual classification on alibrary of spill images, and identify the spill in the at least oneimage based at least in part on the learning-based visualclassification.
 11. A method for detecting a spill comprising:generating a first image of a first scene at a first location thatcontains a spill; generating a second image of a second scene at asecond location that contains no spills; segmenting the first image todetect the spill from at least thermal values in a segment of the firstimage, the segmenting of the first image comprising identifying adifference between (i) a known thermal value associated with the secondscene and (ii) the at least thermal values in the segment of the firstimage; identifying the spill based on the difference; and generating analert indicative at least in part of the identification of the spill.12. The method of claim 11, further comprising adjusting a temperaturevalue of the first scene while generating the first image.
 13. Themethod of claim 11, further comprising determining a confidence value inthe identified spill, wherein the generated alert is further indicativeof the confidence value.
 14. The method of claim 11, further comprisingsensing at least one of reflectance properties, emission properties,electrical properties, noises, and friction of the first scene.
 15. Themethod of claim 14, further comprising determining a confidence value inthe identified spill based at least in part on the segmentation of thefirst image and the sensed at least one of reflectance properties,emission properties, electrical properties, noises, and friction of thefirst scene, wherein the generated alert is further indicative of theconfidence value.
 16. The method of claim 11, further comprisingreceiving an action command in response to the generated alert andperforming an action in response to the action command.
 17. A robotcomprising: an actuator configured to move the robot between a pluralityof locations; a spill detector comprising at least one optical imagingdevice configured to capture at least one image of a scene containing aspill while the robot moves between the plurality of locations; and aprocessor apparatus configured to: generate an action command based atleast in part on the at least one image; receive feedback from anoperator, the feedback comprising at least a confirmation of thegenerated action command; and based at least on the confirmation:perform an autonomous physical action with respect to the spill; adjusta confidence parameter associated with a detection of the spill by therobot; and associate the autonomous physical action performed by therobot with detection of a subsequent spill.
 18. The robot of claim 17,further comprising a temperature adjuster configured to change atemperature value of the scene containing the spill.
 19. The robot ofclaim 17, wherein the action command is a stop command that isconfigured to stop the robot from moving between the plurality oflocations.
 20. The robot of claim 17, wherein: the robot is in datacommunication with a user interface configured to receive the feedback;the action command comprises a query by the robot, the query beingrelated to at least one of (i) the detection of the spill and (ii) theautonomous physical action with respect to the spill.